So today, I had this super interesting session today about AI safety, and let me tell you, it was packed with insights. I figured I'd share some of the highlights with you because why not? Like, we’re building these systems to make life better but we also need to make sure they’re not plotting to take over the world. So this is a big topic and I recommend everyone read this. Adversarial Attacks Okay, so imagine you’re trying to mess with an AI by showing it something it’s supposed to understand but you tweak it juuust enough to confuse it. Like adding tiny noise to an image of a cat and suddenly the AI thinks it’s a dog. This is an adversarial attack. A common one we discussed is FGSM (Fast Gradient Sign Method), it uses the AI’s own gradients to trick it into failing. Or imagine there is a self-driving car and a bad actor figures out how to tweak just a tiny part of your input (like adding a sticker to a stop sign IRL), and suddenly your model thinks it’s a yield sign. Defending against this? It’s hard, but adversarial training (basically teaching the AI with these sneaky examples) helps. But open-source models are a tough nut to crack because attackers have all the access. Makes you wonder—are we really ready for this at scale? And solutions like adversarial training are cool, but it’s a constant cat-and-mouse game to keep up. Like, will we ever really outpace the attackers? IDK, but it’s intense. Machine Unlearning This part was wild. It’s like telling your AI to “forget” specific dangerous stuff. Say it knows how to generate unsafe instructions (like hacking tutorials or something), you can use methods like RMU (Representation Misdirection for Unlearning) to wipe out that knowledge while keeping the useful stuff. But also… super delicate. Mess it up, and you might “unlearn” things you actually need. The balance here is insane. Still, the concept of RMU was just *chef’s kiss.* AI Control Protocols This was probably my favorite. Imagine you have a sneaky, super-smart AI (untrusted model) and you pair it with a safety buddy (trusted model). The buddy’s job is to double-check outputs, flagging or fixing anything harmful. Two approaches stood out:

Hello Rishab Academy community & Rishab This is something i’ve been contemplating about recently. There is a well-known Scence and Math academy high school that I can attend. I dont know if i should, as my goal is to get into MIT/Standford/Harvard. I also am interested in doing research, and they have a research program. I’m worried about competiveness, as this is also a feeder school to MIT. I feel that there will be students way more talented than me over there that will take my possibility of being an H/S/M student. What do you think? The course rigor there is also alot so that will leave me with even less time for research and other ec activies. Let me know what you guys think

Excited to announce the International Research Olympiad has released our official textbook. The International Research Olympiad Textbook is a comprehensive resource crafted for ambitious students ready to tackle the unique challenges of the IRO. Unlike traditional science olympiads, the IRO emphasizes cultivating versatile research and analytical skills that are applicable across diverse scientific disciplines. This prep book, developed by the IRO and supported by dedicated professors offers detailed content reviews, targeted practice questions, and full-length mock exams. Alongside the textbook, students gain access to additional online resources, such as daily problem challenges and mock exams, ensuring effective preparation. The opens exam contains 4 research papers and a total of 32 MCQs, followed by a semifinal exam, and an in-person final exam. Purchase the textbook on Amazon for $19.99 here: https://amzn.to/4inoYWH

He everybody, I was curious and I thought I might just throw it out there. Is anyone here interested in chess, if so please drop your name and anything you like about the game e.g opening, fav player or YouTuber or favourite ROOOOOOOOOKKK! Sacrifice

Hi, where does everyone buy their wet lab supplies from likes beakers, test tubes etc for experiments? I am in middle school.