Hi, @Shyon Babazadeh ! I have been using it at work just to ask questions to for my job. Boring stuff like "how do I make the top row in my Excel spreadsheet not scroll". It seems fast and competent. But I haven't used it for prompt engineering/apps yet. There's part of me that just wants to stay away from OpenAI for a long time because they've subsidized themselves to bring token pricing down below their own costs. I'm worried that they will operate at a loss or flat-margin until their competitors are gone. And then, after that, they raise prices or perhaps stagnate in comparison to what their competition could have provided. The same kind of criticism/concerns could be applied to other large GenAI players' foundational models too. That's part of the reason I'm focused on LLMs that can run locally. So I just have a hard time getting excited about a cheap, competent, black-boxed model from OpenAI.

I went to the one yesterday and met Evan and Brian. Really recommended!

Best practices for model-specific tool calling (aka function calling). I have found a lot of things that seem like they should work, but in practice, fail miserably. So for example, getting Llama3 to make a tool call and not be confused by the response coming back. There are some models that seem optimized for tool calling and others that are just bad at it. It's really nice to know if a model is just really shouldn't be used for tool calling. I've also seen workarounds that skip tool-related messaging, and give better results in some cases. But this is an area that just feels like the documentation/how-to guides haven't been properly written yet. And maybe the right kind of topic to apply Evan's great hands-on approach to.

My personal tastes cause me to skip over any definition of "consciousness", "awareness" and philosophic implications. The more interesting thing to me is this idea of continuously updating information for the agent, some of which may or may not be actionable, and seeing how the agent can act usefully on that information. Think about how this is different than typical RAG. With RAG, you'd be retrieving contextual information that mostly correlates with some kind of chosen goal or action, e.g. retrieve the current temperature in Maui because that question came up. And I may misunderstand Brian's idea, but I see it as different than RAG or at least it's an atypical flavor of it. The context window is updated regularly with new information, some of it superfluous, and you are asking the LLM on some cadence, "with all that you know, and maybe hoping to achieve certain goals, what do you want to do or say now?" There might be advantages to having a statically structured "awareness" section of the context window rather than a rolling window of new information deemed relevant. For example, your system message might contain instructions that reference constant data items in the context window like "last image from the camera" and therefore work more reliably as these things are guaranteed to be in context and their position relative to other pieces of information can be constant. And my thoughts here are vague, but... maybe some advantages in fine-tuning? Like maybe you can train with the assumption baked into the training that certain pieces of data will always be coming from context. Potential disadvantages: maybe the LLM has less clues about what information will be relevant to its goals, so it gets lost easier. (This is the same problem as throwing a giant doc at an LLM, and it's not good at using all the information.) And it might be an inefficient use of limited context window size, if you have a lot of Awareness data to populate. Brian, my apologies if I just went off on musing that weren't close to what you meant. Thanks for the thought-provoking idea.