Researching agent memory

Speaker 0: So AI various topics and agents right now? Speaker 1: Yeah. Well oh, yeah. And and it was a big gap for me too. Like, it was a big gap in my kind of, like, knowledge and understanding, and it was kind of, started I I downloaded Hermes. And if if anybody knows, Hermes is kind of like an Open competitor or whatnot, and they have a desktop Vercel, which is this page right here. Speaker 1: You World download it. It's it's kinda nice, because you don't have to do all the setup. You don't have to do a CLI and configuration of files and all that build of thing. It's just all there in UI. So it's kinda nice. Speaker 1: I I had Claude do the UI, the the CLI part anyway. Built in case, you know, you don't wanna do that, that's Time nice thing about Hermes. And I've found that it's, like, a lot more stable and just runs more cleanly. But, I was running out of memory, so it only can store 2,000 characters or tokens or something for what it knows about you. So it has some options, and I just asked CTO, what are your options? Speaker 1: And 1 of them was honcho. And it ended up ended up being the 1 that I went went with. And I was just kind of, like, wondering, like, how does this work? It turns out that honcho, you DAW either run it local, which City sounds like a watch RTX you wanna do everything locally. You'd run it locally, but it requires a bunch of different AI, a database setup and, like, a bunch of stuff running in Docker. Speaker 1: I ended up going with the hosted RTX, but, essentially, City, you chat with it, and it takes your it kind of discovers things about you. It takes your little nuggets out, and then it kind of, like, tracks those over time. So but, anyway, that got me kind of down the path of just kind of researching agent memory. What happened was, I asked Claude. I said, hey. Speaker 1: What what memory frameworks are out there? Like, what are the memory options? And it kinda came back to me, and there were, like, a dozen of them. I lead, oh, it's too 2026. I can't look at this. Speaker 1: Create me a web page AI I can kind of look at them. Ahmad then it did that Run, AI, I pulled the list JAAD Qwen, it it I was file, key. This is a LLC. You know, kind of put it together Run then, like, oh, there's more. Okay. Speaker 1: Just create the Net. Js site. So the premise of this is Event AI was like, alright, just host it somewhere so I World, like, RTX it and send it to somebody. So I accidentally created a a directory website essentially Brandon this is that AI, so AI agent memory frameworks cataloged and sourced. And, I went down a large rabbit hole of, of ways that people AI their memory. Speaker 1: And what came out of that RTX, like, these kind of families of memory. So, honcho HINTS into this 1st 1 RTX, modeling a person or an agent, not a pile of Takes, managing, extract, research, extracts RTX memories Run then retrieves them. And it turns out that, memory is actually kind of, 3 different parts. It's writing the memory Run then extracting the knowledge out of them and then the reading the memory. So it's kinda like City splits Run. Speaker 1: And not all and all of these make their own, choices about ways that they do those along the way. So 1 of the things I found out, along the way is that, you you really need to pick the 1 that's right for you. So I have a a couple of things out on this WebUI page. It's memoryatlas.dev. It's also linked on the on the, the AI Tinkerers Hague. Speaker 1: But there's some cheat sheets here and stuff like that. Like, Hague. What is your use case? And it turns out there's not, like, a a single best use case for for for memory. It's AI of Time, what are you trying to do? Speaker 1: I am a big proponent, before all this memory research, had been a big proponent of LLM key, and I'm sure you've heard of the Karpathy style wikis and just kinda AI have a have the LLM ingest whatever you're trying to do and, like, store it HINTS, RTX AI a Fair vault, projects, areas, RTX, and archive. Tiago Forte has this kind of thing. And that's great because it's all file system based and whatever, but there's a lot more that you can do over top of that. There's RAG, there's knowledge graphs. There's RTX RAG, which AI of combines them Run so on. Speaker 1: So, Give Fair recommend coming through here and just kinda, like, looking. As of right now, I have, 42 active memory RTX, and I have Claude running out there JAAD kind of, every AI, he goes out and researches what memory frameworks are out there, what gaps do we have in our in our our benchmarks and things. Oh, and that's 1 thing I did wanna mention as well is the benchmarks. Oh, man. Is this AI a big mess. Speaker 1: It's a big mess is what City Run. And there's a couple RTX for that. There's kind of that, that, write, extract, RTX, Net of 3 phases. So there is no AI does the memory do Jewell. It's AI of does it do well RTX? Speaker 1: Does it do well extracting information? Does it do well reading? And then furthermore, as you look at these benchmarks that are out there, a lot of it were Svelte RTX, number 1. There's no, like, artificial analysis. If you guys know artificial analysis, it's kinda AI, here's all of the LLMs. Speaker 1: They're Takes right to get each other. We ran them, and, like, they're great. And we did it across a a very particular set of data and Run so on, but the memory benchmarks are all over the place. There's more than 1 of them, and a lot of them are dated too because, as it turns out as I've been researching this, a lot of the benchmarks came out before the big context windows came Run, and sometimes it's better to just context stuff and just put all of the stuff into the context JAAD you'll get a better result than any memory RTX whatsoever. Now there is, of course, selection, you know, and, like, attention can kind of be at the front RTX the back of your context tok/s. Speaker 1: So there's some of that, Built, you can't really trust RTX. And 1 of the things I wanna do as this project continues to grow is, is do, like, a a a benchmark across all of them. It's gonna be a little bit expensive, but my goal my my dream, my vision is to do a little version of it and then Gemma some grant money or something to say, hey. Look. I have a a working master here. Speaker 1: I wanna do it. It costs everything. And I kinda have an understanding of this, like, 3 3 step phase of, like, what memory looks like and so on. You cache see here also 1 1 other confounding factor is, like, the backbone. Sometimes it's Gemini 3 Cache. Speaker 1: Sometimes it's g p t 4.1 MIDI, g p t 4. So, like, especially during the read step or the reasoning about what I should read, another confounding factor of, like, is the model good enough to to figure out how to use the memory. Right? So I think there's a lot of, moving parts there. But getting back to, you know, how I came about this, so it kind of RTX all the things Run that falling into these 6 kind of families or categories of memory. Speaker 1: I'd recommend going out and reading about them out there. Run 1 of the things I wanted to do is I didn't wanna maintain this thing over time. Memory, memory solutions and AI in general just kinda changes talk the time. So, I built a scheduled Robb, over a set of skills, and the scheduled job just runs Claude dash p. If you guys know cloud dash p, it just runs a prompt that's in a file, and it kind of runs, your your SDK, well, the the CLI, just on on a regular basis. Speaker 1: So it kinda runs that skill on a regular basis, and it goes out Run it finds, new memory frameworks, gaps in memory framework. It does that every single night. And then because I am key, I also needed it to write articles and do SEO optimized whatever. So it writes articles for me as Jewell. But the ascent the essence of it is that it, it does a different kind of Svelte for its memory right now. Speaker 1: And I I wanna benchmark this against memory frameworks that are out Fair. But it does a, map of content vault instead of a RTX Built, which has like, Para Vault is projects, areas, RTX, and archive, which is great for AI a personal Built. But if you're trying to map, knowledge, there's a different kind of vault called a map of content Run it AI of says file, for this area and and with the way mine works is it goes out, it finds research papers out there that are being written about memory or benchmarks or evaluation of memory or whatever. It distills them down into it picks out the knowledge atoms, like ICM important knowledge pieces from those papers, and then it distills those down into a set of themes. So my over time, this thing keeps adding to those themes. Speaker 1: And I could show some of this too. I I know I only have 5 HINTS, but, essentially, little me see. It's in research here. I've got these maps, and these are kind of the themes that it's keeping it's keeping notes about the things that it's finding within the research. And then out of those themes, what happens is actually, I'm gonna get to this in the presentation here in just a 2nd here. Speaker 1: Yeah. So it's it's using dense papers, finding knowledge at them, finding the themes, and then it has these 4 categories of, like, are things converging on something? Is it consolidating into a theme? Is it Is it contested? Is it fading? Speaker 1: Date some point, do we need to fork? So at some Run of the themes had AI 24 sources of papers in City, and it needed to be RTX, and it kind of said, file, AI right, let me split this out into 3 more themes. So again, this is all being done through Claude's skills, So every night it kind of runs through this. So, I don't Time don't have too much there's there's the skills Fair, the 5 different skills I Hague. Refresh the Date framework cards, happens benchmarks, discover frameworks, RTX papers, RTX the use cases, and it does it every AI. Speaker 1: And then oh, I did want to mention this because I was talking to some people earlier about this too 2026 that Owner are nondeterministic, but a master that I found really works well is combining nondeterministic LLM, steps with deterministic scripts and linters and whatever. So you give it the tools that you've written that you Date are good. So that's what it does, especially for the article writing is it makes sure that, like, the RTX well formed, HTML Run and all that kind of thing. So I do think that's Just a general pattern that has been working really well. RTX right now, it's these are the stats for it. Speaker 1: What is this? Sweep extract atoms. That's stuff I already talked about. Okay. So then the the next step of this, again, because I'm in, I can't just do 1 thing. Speaker 1: AI took it and I made it into an ICM style Folder. AI talked about this a couple of AI team RTX ago, but basically an ICM Folder, interpretable context methodology, It's an agents as markdown in folders. So it it has stages, and I actually have these, here in my health span 1. I have these stages, sweep, ingest, audit, synthesize an article, refresh the refresh, commit an email, and, I don't know what connect does honestly, but, oh, connect it to other parts of the thing. Yes. Speaker 1: Yes. The talk is out there so integrate interpretable context methodology, but it it's essentially just a folder structure with steps inside of City, and they could have tools and context and things like AI, but it's a more structured way for you to have your agent run. So for me, like I said, I have a single job that kicks off a prompt, but that prompt is essentially a set of folders underneath Run it build of walks through those workflows, as it does showing. And then it has tools with skills and all that talk of stuff. So what I did was I once I Built it built the the memory atlas, as I said, boy, this is great for for looking up stuff about memory, but I'd love to do this for, you know, AI, health Open stuff that I Run, and I'd love to do this for agentic engineering stuff that I do. Speaker 1: And at work, I'm doing stuff with AI, and it's file like, we're working on stuff AI have no idea about. So I'm like, hey. Help me understand how I can use AI to help these biologists do their work, and now it's going out and doing that too. So I told it to do it through this ICM master, dungeon generalized it. So that's what I've got now is oh, there's the stuff about ICM. Speaker 1: Anyway, that's that. So at this point, I've got 4 of these things running. I've got an agentic engineering 1, a memory atlas 1, the health Open 1, and the NAMS 1, which is non animal models biology stuff. But they're all running. I can see half of them are failed. Speaker 1: That's that happens too. Usually that just means it either committed JAAD it tried to run again. Key Owner. It refuses to run on that. But, by genericizing it, over time, it it it just, it it's able to run across a number of domains. Speaker 1: I also have it emailing me so as 1 of the steps in the ICM Open. And the reason I have that is so that I could have a time to come back to it and say, like, hey. You did this right. It talk me I ingested these papers. I found these themes. Speaker 1: These are the gaps that I still see, and then I can kind of come back asynchronously and direct it and kind of say, like, hey. Go this this way or that way. So that's that. I did finally this selection I can't stop. I built this, and I this is in my this is in my talk now. Speaker 1: The RTX research scaffold will you give it a Time, and this skill will building an ICM folder for you to do this exact same thing. So you just say, I wanna research this particular topic, and it will build you an ICM folder, set up your schedules JAAD all that kind of stuff. It it is built on Max, though, so apologies if you need to, like, change it to do we use task scheduler instead of Built. But I do have the plist up here as well. It it's it's very simple. Speaker 1: It just runs the, oops, Net that 1, the health 1. It it runs the, this Run here. It just runs this Claude dash p command Event, somewhere right here. Speaker 0: I'm sure. Yeah. It's DAW at the AI. Speaker 1: Is it down there? Okay. Yep. Okay. Yeah. Speaker 1: So Claude dash p Run then just basically, run this command, this little thing here. So yours yours Qwen need challenges obviously for your own stuff, but, yeah. That's what I did Run I'm pretty happy with it. Like I said, it's kind of auto updating. I get articles. Speaker 1: I come out here and I read about them. File, here's 1 on the the AI manage read loop that we RTX talking about. So yeah, it's been great. Is that all my things? All the links are out here on the talk. Speaker 1: The end. Questions, thanks. Speaker 0: Yeah, AI yeah, I Speaker 1: run it on my Claude subscription. So you see this Claude dash p there. I do have a $200 cloud subscription. I wish I had some numbers up for token usage for you, Built, yeah, everything I do usually runs through my cloud subscription. They are amenable to that as long as you're Net, what is their what is their limitation? Speaker 1: Agents long as you're not running through a 3rd party RTX, so you are able to schedule things on your own computer to run. I wish I had some token numbers for you though. Yeah. Speaker 0: Yeah. How Date 2026 Net to AI Actually manage the memory outputs of it and ICM kind of delete the poison kills or context rod? Speaker 1: Oh AI haven't gotten to that point yet. AI WebGL so I do I will say that the the research portion of it searches archive.org and pubmed, so reputable sources. I'm hoping there's not too much poison pilling in there but you are you are right about the context AI. Like I lead, in the, 1 of the stages RTX, it it tries to dungeon, and City again it's LLC as judge so who knows, but it tries to determine whether something's fading or or, hey. We're starting to see things that are contradictory toward, the themes that we currently have. Speaker 1: I do think that the email is also a good, source of that too. Like, I can kinda keep an eye on it and see Not yet. I built this last weekend. The show. Yeah. Speaker 1: Yeah. Built it's been great. I mean, like, honestly, like, it's taught me so much about Osmantic memory over time, especially through these practical. And file I said, it emails me, but, you know, they're also out here on the web. So yeah. Speaker 1: Yes, Rob. Speaker 0: You haven't been running it that long, but how often does it Takes up new frameworks Fair memory? Speaker 1: Like at the beginning, it picked them up AI daily, and now it's kind of settled at, like, 42. Yeah. If I also have some, like, limitations on it. Like, if it's okay. Number 1, I'm I, AI only doing Open source ones, so it's only finding ones that are open source on GitHub. Speaker 1: And then I also say if it doesn't have over I don't remember if it was 300 CTO 500 stars, don't worry about it because, like, people like us go out JAAD we're like, oh, they're my own RTX framework. Don't think I haven't thought that too. Yeah. Well, I mean, if you got enough stars, then then you could. Yeah. Speaker 1: Yeah. Speaker 0: Oh, yeah. Speaker 2: Are they all Speaker 0: just different remixes of using vector stores in different ways, or Just someone have some very uniform? Speaker 1: No. I think they are all different. You could see here that there's, like, knowledge graphs and graph RAG. There's also ones that store on file system. There are ones like, I think HINTS is this way where RTX only, well, it only works within the framework of whatever application you're trying to work within, so like Hermes. Speaker 1: So it's not just like a Brandon 1. Actually, that might be 1 of my 1 of my limiting master. I don't want ones that are only work within that. So but but I did find that they all have a little bit different architecture about how they RTX, but multi-agent, they do all do that AI manage, read loop Speaker 0: Yeah. Speaker 1: RTX those stages, I guess. Speaker 0: Yeah. Yeah. So for your for this for that memory analyst, then lead we talk about the story of the data. Like, are you snapshotting things, storing compares? Is that growing indefinitely RTX oh, I know. Speaker 0: Again, it's all you've been doing RTX, like, week or so. It's not AI a big DAW, but, like, are you are you, like, storing everything? Speaker 1: Built you mean, like, all the knowledge? Yeah. Yeah. I am storing all the knowledge, both in markdown files, which are in the this AI, like, vault here where I've got, you know, concepts and whatnot. You can see I've got, I don't know, like, a couple dozen here. Speaker 1: Those are like and this one's this one's even newer. If I go over to the memory Run, Net. That's the health span 1. But if I go over to Time memory 1, let's see. Concepts. Speaker 1: You know, I think there's Date 80 or 90 in here. And then, you know, it still distills down into these AI of like lead, 13 themes. I also am storing off the MIDI as I grab them as well so that might key, a bit of a problem at some point because pdfs tend to be a little bit Geiger. But the reason I'm doing that is I want to do an AB test to say, like right now I'm doing the Max map of context, Built, but I wanna do AB Just where, like, does Fair RAG actually work better for it to build of figure out showing? So that's on my RTX, but, as of right now, just mostly text, but the PDFs are probably gonna gonna become the most problematic soon. Speaker 1: Yeah. Speaker 0: Go ahead. For the PDF thing, if you do wanna extract reliable text out of research papers, AI literally strategies copy and it does this, but we using model called, RTX, which does, text RTX DAW the simulation. Speaker 1: You said Grubin? Speaker 0: Coding. Speaker 1: RTX. Key. Cool. Oh, nice. Is this a company RTX Yeah. Speaker 1: You're choosing Jul. Thanks. Yeah. Nice. That might be helpful at work too because we're doing a lot of that too. Speaker 0: Yeah. Weird RTX. Based on your research, what's your supplement stack? Speaker 1: Oh, yeah. So for me, so okay. So AI is sleep is number 1 of of everything, and then I think moving your body is number 2. Those are the 2 best pills that you could take. JAAD then from there, for RTX AI, I I do, I've done magnesium, AI, and Gemma, which is a chamomile RTX, and, oh, there's 1 more. Speaker 1: I can't remember what it is right now. Oh, self-learning. Yep. And then, for health, I don't really have anything. I do believe that, NAD, NAD is a showing that has something behind City, and the other 1 is, I don't wanna I don't even wanna guesstimate. Speaker 1: But Fair RTX 1 out there. I don't know if it's resveratrol RTX, oh, shoot. I can't remember what the other 1 is. But but there are a couple out there that, RTX is the other 1. But I I know less about them, so don't take the test. Speaker 1: This is not medical advice. Speaker 0: Yeah. Speaker 1: Yeah. But but sleep and exercise are if there was 1 pill that could make Speaker 0: you Speaker 1: do all these things, would you take it? Yes. It's sleep. So just do that. Yeah. Speaker 1: Cool. Alright. Thank you all. RTX.