Discussion about this post

User's avatar
wioc's avatar

Thanks to Brent Donnelly for the reference. I wonder if the solution to the optimization danger is a fuller context for the agents. For example, humans can play sports and other games within established rule sets. While a player may break the rules if they can do so without being noticed, there are clear limits on acceptable behavior (e.g., bringing a gun to a tennis match ruins the game for everyone, and bringing a tennis racket to a war zone is...inadequate). Why can't we train our LLMs to play games according to pre-defined context that includes all the behaviors we wish everyone would use in their daily life? By the time we're done programming the context we'll probably end up with a bunch of things like the golden rule and then the LLMs will be helping us live up to our own potential rather than accidentally endangering us.

3 more comments...

Ready for more?