Gemini 1.5's groundbreaking 1M token context window is a remarkable advancement in LLMs, providing capabilities unlike any other currently available model. With its 1M context window, Gemini 1.5 can ingest the equivalent of 10 Harry Potter books in one go. However, this enormous context window is not without its limitations. In my experience, Gemini 1.5 often struggles to retrieve the most relevant information from the vast amount of contextual data it has access to. The "Needle in a Haystack" benchmark is a well-known challenge for LLMs, which tests their ability to find specific information within a large corpus of text. This benchmark is particularly relevant for models with large context windows, as they must efficiently search through vast amounts of data to locate the most pertinent information. To address this issue, I have developed a novel prompting technique that I call "Multi-Candidate Needle Prompting." This approach aims to improve the model's ability to accurately retrieve key information from within its large context window. The technique involves prompting the LLM to identify 10 relevant sentences from different parts of the input text, and then asking it to consider which of these sentences (i.e. candidate needles) is the most pertinent to the question at hand before providing the final answer. This process bears some resemblance to Retrieval Augmented Generation (RAG), but the key difference is that the entire process is carried out by the LLM itself, without relying on a separate retrieval mechanism. By prompting the model to consider multiple relevant sentences from various parts of the text, "Multi-Candidate Needle Prompting" promotes a more thorough search of the available information and minimizes the chances of overlooking crucial details. Moreover, requiring the model to explicitly write out the relevant sentences serves as a form of intermediate reasoning, providing insights into the model's thought process. The attached screenshot anecdotally demonstrates the effectiveness of my approach.

Hello! Check these exciting new AI updates for today: 𝗠𝗨𝗦𝗧𝗔𝗙𝗔 𝗦𝗨𝗟𝗘𝗬𝗠𝗔𝗡 𝗝𝗢𝗜𝗡𝗦 𝗠𝗜𝗖𝗥𝗢𝗦𝗢𝗙𝗧: Mustafa Suleyman, co-founder of DeepMind and Inflection, has joined Microsoft to lead Copilot. With his extensive background in AI, his role at Microsoft promises to bring innovative advancements to AI technologies. Discover how his expertise will shape the future of AI at Microsoft. Read more 𝗥𝗢𝗕𝗢𝗧𝗦 𝗥𝗘𝗣𝗟𝗔𝗖𝗜𝗡𝗚 𝗖𝗜𝗩𝗜𝗟 𝗦𝗘𝗥𝗩𝗔𝗡𝗧𝗦: Explore how AI is transforming civil service jobs by automating repetitive tasks, thereby reshaping the workforce landscape. As AI technology continues to evolve, its applications in various industries are expanding, paving the way for more efficient and productive work environments. Read more 𝗬𝗢𝗨𝗧𝗨𝗕𝗘 𝗖𝗥𝗘𝗔𝗧𝗢𝗥 𝗦𝗧𝗨𝗗𝗜𝗢 𝗟𝗔𝗕𝗘𝗟𝗦 𝗔𝗜-𝗚𝗘𝗡𝗘𝗥𝗔𝗧𝗘𝗗 𝗖𝗢𝗡𝗧𝗘𝗡𝗧: YouTube Creator Studio introduces a new label to flag AI-generated or altered content, addressing concerns about misinformation and authenticity. This initiative aims to enhance transparency on the platform and provide users with more context about the content they consume. Read more 𝗠𝗜𝗖𝗥𝗢𝗦𝗢𝗙𝗧'𝗦 𝗦𝗣𝗘𝗖𝗜𝗔𝗟 𝗪𝗜𝗡𝗗𝗢𝗪𝗦 𝗔𝗡𝗗 𝗦𝗨𝗥𝗙𝗔𝗖𝗘 𝗔𝗜 𝗘𝗩𝗘𝗡𝗧: Microsoft is gearing up to host a special event focusing on Windows and Surface AI innovations in May. With the integration of AI into Windows and Surface devices, users can expect enhanced functionality and productivity. Explore what to expect from this anticipated event and how AI will shape the future of these technologies. Read more

## The Devin Hype Devin has been hailed as a milestone in the evolution of AI, showcasing traits of AGI specifically tailored for software engineering. Its ability to demonstrate high levels of autonomy and adaptability, including debugging, learning from documentation, and applying fixes independently, has captured the attention of the tech world. Devin's successful performance in engineering interviews and real-world tasks on platforms like Upwork has further fueled the excitement, suggesting its readiness for practical applications. One of the most impressive aspects of Devin is its capacity to learn autonomously from new sources, such as blog posts, and apply that knowledge to tackle novel challenges. This advanced level of comprehension and application hints at the potential for Devin to significantly impact the software development industry by automating tasks and enhancing productivity. Devin’s proficiency in executing complex projects, from web development to setting up computer vision models, and its seamless integration of various tools to mimic human engineering workflows, further underscore its sophisticated capabilities. ## Criticism and Skepticism Despite the excitement surrounding Devin, critics have raised significant concerns about the hype and the validity of the claims made by Cognition Labs. Skeptics argue that Devin's functionalities are not entirely unique, and that similar outcomes have been achieved using existing AI agent frameworks such as AutoGen, CrewAI and ChatDev. They have demonstrated that many of the features showcased in Devin's demo can be replicated using the ChatGPT API and basic coding skills, questioning whether Devin truly represents a quantum leap in AI's role in software development. A closer examination of Cognition Labs' website and the preview URL for Devin has revealed several red flags that suggest the company may not be as sophisticated as it claims. For instance, the website itself appears to be of poor quality, raising questions about why Devin, if it is truly capable of advanced web development, has not been utilized to create a better site. Furthermore, the preview URL for Devin looks vastly different from what is shown in the promotional video, casting doubt on the authenticity of the showcased capabilities.

This is the most important AI development of 2024 so far. This is bigger than Sora, bigger than Gemini 1.5. This is Devin, an autonomous AI software engineer agent that actually works. Please don't sleep on this and look into it. My words don't do it justice. Andrej Karpathy, who recently left OpenAI and is perhaps the most important AI developer of his generation, does a much better job explaining the sheer magnitude of what has happened: https://x.com/karpathy/status/1767598414945292695?s=20 People, it's completing freelancer tasks from Upwork, autonomously, by itself. Things are about to get very weird. There is a wait list. Sign up today. Blog post: https://www.cognition-labs.com/blog Wait list sign up form: https://forms.gle/PJPKaKYRZv9jfXP6A