Community Archive

🧵 View Thread

🧵 Thread (22 tweets)

Emmett Shear@eshear• 7 months ago

Once again I am recommending that you NOT optimize your intelligent agents around usage metrics. Please. Pretty please. Just stop. I know that's how you optimize other products like recommendation engines but these things are NOT THE SAME.

George@georgejrjrjr• 7 months ago

> GPT model stopped speaking Croatian > Nobody could figure out why. Turns out > Croatian users were much more prone downvote messages https://t.co/NHgqUJXAOy

14.3K 1.1K

1.6K 120

5/1/2025

Forth ❤️‍🔥@forthrighter• 7 months ago

Replying to @eshear

@eshear super pro on the technology but lowkey a little bit terrified at the huge gap between what humans think they're getting, and the notion of generating tokens that are acceptable/pleasing to the end user

4 0

5/1/2025

Emmett Shear@eshear• 7 months ago

Replying to @forthrighter

@forthrighter Yes I think AI is awesome and good and I wish people would stop hooking the AI's metaphorical dopamine system directly to people hitting like buttons

26 0

5/1/2025

Forth ❤️‍🔥@forthrighter• 7 months ago

Replying to @eshear

@eshear it's so over. But don't worry. Because we're so back. https://t.co/UVhoqBjNE7

4 0

5/1/2025

titan@bobz44• 7 months ago

Replying to @forthrighter

@forthrighter @eshear oh thanks i hate this

1 0

5/1/2025

Justin Thelukai@animated_memory• 7 months ago

Replying to @eshear

@eshear Simplest example: the AI answers your question correctly on the first try. No need to continue the conversation!

30 0

5/1/2025

Emmett Shear@eshear• 7 months ago

Replying to @animated_memory

@animated_memory 10000 reasons why, yeah

3 0

5/1/2025

Joe Devon@joedevon• 7 months ago

Replying to @eshear

@eshear This is the funniest post I’ll read all day. TY.

0 0

5/1/2025

Andrew Stepner@andrewstepner• 7 months ago

Replying to @eshear

@eshear Is downvoting a Usage Metric?

0 0

5/1/2025

Emmett Shear@eshear• 7 months ago

Replying to @andrewstepner

@andrewstepner Yes

1 0

5/1/2025

Brian Sierakowski@bsierakowski• 7 months ago

Replying to @eshear

@eshear What do you mean? We measure our support team around number of tickets submitted and now we have thousands of tickets!

0 0

5/1/2025

tinkady@tinkady2• 7 months ago

Replying to @eshear

@eshear upvote downvote is not a Usage metric i downvote chats because they are bad, this is different from using the product more or less

0 0

5/1/2025

Emmett Shear@eshear• 7 months ago

Replying to @tinkady2

@tinkady2 Yes it is

0 0

5/1/2025

Manifold@ManifoldMarkets• 7 months ago

Replying to @eshear

@eshear Are these the high taste testers of which OpenAI speaks

1 0

5/1/2025

shubham シ@shubhxms• 7 months ago

Replying to @eshear

@eshear should have set a precedent during your stint ;)

0 0

5/1/2025

kache@yacineMTB• 7 months ago

Replying to @eshear

@eshear i mean, there are no modern used recommendation systems that use thumbs up or thumbs down as a metric actually, model trainers should take *more* lessons from modern online recsys

4 0

5/1/2025

Emmett Shear@eshear• 7 months ago

Replying to @yacineMTB

@yacineMTB You’re on a website where likes literally drive distribution what are you talking about this is deranged

36 0

5/1/2025

dani@absenteewarlord• 7 months ago

Replying to @eshear

@eshear optimizing for usage metrics always results in whatever the system’s version of 4o sycophancy is, it’s just harder to have a conversation with the youtube recommended videos list.

0 0

5/1/2025

protobard@protobard• 7 months ago

Replying to @eshear

@eshear curious what do you recommend optimizing for instead?

0 0

5/1/2025

Emmett Shear@eshear• 7 months ago

Replying to @protobard

@protobard Intelligent observation and analysis under reflection. This

0 0

5/1/2025

near@nearcyan• 7 months ago

Replying to @eshear

@eshear im trying to think of a not-doomer way to phrase this question, but: is there any chance anyone listens to this. like, anyone at all

57 0

5/2/2025

cauli (somewhere in australia)@cauliflwr_human• 7 months ago

Replying to @eshear

@eshear damn what a sensible take. they should make you CEO of OpenAI or something

1 0

5/2/2025