📗 -> 04/02/25: ECS170-L2

[Lecture Slide Link]

🎤 Vocab

❗ Unit and Larger Context

Small summary

✒️ -> Scratch Notes

ChatGPT

Pretraining, predicting next token
Instruction tuning, Fine tune on different tasks
RLHF - Reinforcement Learning from Human Feedback
In Context Learning

Few Shot

Provide task description, but also show a few examples of the task

Translate English to French:
sea otter -> loutre de mer
peppermint -> menthe poivree
stuffed giraffe -> ---- ?

Problems still:

Hallucinations
Reasoning is not rigorous, makes simple errors

Outline

What is an agent?
What is a rational agent?
How to build a rational agent?

Agents

An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators

Sensors and Actuators are 2 important components to agents

Humans are agents!

Sensors: eyes, ears, …
Actuators: hands, legs, …

An Agent is a Function that Perceives and Acts

Percept: Perception inputs at a time
Percept Sequence: Vector P of perception history
Action set: A
Agent Function: f: P -> A
Agent Program: A program that implements an agent function
Architecture: the computing device with physical sensors and actuators that the program can run on
Agent = architecture + agent program which implements the agent function

Rational Agents

Rational Agent - An agent that does the right thing, meaning that the agent function is defined correctly in the table

But how to define?
Performance Measure - An objective criterion for success of an agent’s action given the percept sequence
- A performance measure for a vacuum-cleaner agent might include:
  - +1 for each clean of a square in time T
  - -1 point for each move to a different square because of electricity and noise

What is rational at any given time depends on four things:

Agents prior knowledge of the environment
Actions that the agent can perform
??

Design Process for a Rational (AI) Agent

Precondition: specify a task environment for the AI agent
Design: Construct a function f to maximize the EV of the performance measure
Implementation: Write and test an agent program that implements f on a particular architecture
- Agent program: Map from percepts to actions
- Architecture: Computing device with percepts and actuators

Specifying Agent Environment

Task Environment: Problem specs for which the agent is a solution
PEAS - How to specify the task environment

Performance measure
Environment description
Actuators
Sensors

PEAS of a Autonomous Taxi

Safe, short travel time, …
E:
Roads, traffic, …
A:
Steering, accelerator, …
S:
Cameras, sonar, speedometer, GPS

Seven Properties of Task Environments

States observability
Number of agents
Successor state determination (deterministic, non, stochastic)
Episodic or Sequential
Environment static or dynamic during decisions
State of the environment and action of the agent over time, discrete or continuous
Environment known or unknown to the agent

🧪 -> Refresh the Info

Did you generally find the overall content understandable or compelling or relevant or not, and why, or which aspects of the reading were most novel or challenging for you and which aspects were most familiar or straightforward?)

Did a specific aspect of the reading raise questions for you or relate to other ideas and findings you’ve encountered, or are there other related issues you wish had been covered?)

Vault

Explorer

ECS170-L2