Generative AI for Fun and Little Profit

Speaker 0: We're good. Okay. Speaker 1: Alright. So a couple years ago, I started playing with, local AI because I just wanted to learn how to do it myself. This is going to be a technology that stuck around. I wanted to do it myself, I wanted to understand it, and I didn't want to have to rely on third parties. A lot of my technology stacks at home kind of revolve around that Context. Speaker 1: Technology independence Sai far as I can as I'm able to do it. I also, you know, once I started doing that, I also wanted to share what I discovered and created with friends and, because I thought that would be, you know, May that'd be really cool if I could just Little actually show people some of the stuff I actually am capable of doing. So I don't really have Time a lot of visible or high visibility projects. So I created MuseBot which is a generative AI, bot for Discord, where all my friends and I hang out. And, it started as, just a fork of an existing project that worked with large language models in Discord. Speaker 1: And then I was Time, May, well, now I wanna do images too. And then I was wondering, well, how many types of media can I make it do? So today, MuseBot can act as a chatbot, image generator, music generator, video generator, and it has some cool image to image extensions, To just works Time through a plug in system I developed. There's a lot to show, and I really wish I could oh, I yeah. This is just the settings file, from Musebot, which is an ENV file. Speaker 1: I have a documentation page that actually might be better at showing this, and I will also zoom in on this. Musebot, you decide if you want it to be a chatbot or produce media and then it determines what type of media it can do based on the COMF UI workflows and its configuration directory. And if you're not familiar with COMF UI it is a node based editor for generating media. It can also do some LLM stuff. But all these nodes together you get a picture of the ocean which is what I asked for. Speaker 1: You can export these from Comfy UI as an API JSON workflow. And with MuseBot, you actually get there's an extension to the comfy Sai workflow where you provide some default parameters, that it will actually inject into the workflows using mustache templates. So I have some example media I've produced with Musebot just to give you an idea of what it's capable of. There's a wallpaper Profit, which generates 1080p wallpapers which can be upscaled to 4 ks. These are just some of the examples of what I've been able to create with MewsBot. Speaker 0: What types of prompts are you using? Speaker 1: So you can, do your own Profit, and it's really it's it's been a 2 year project Sai the prompts are all over the place, But there's also an LLM assistance Profit button that you can click to have it help you out. And just by throwing a lot of detail at these image generation models, that tends to help them out a little bit. Speaker 0: Something else that's but I like to do my server. I'll put in a prompt to get an image, and then you can export the JSON, and then you can fine tune it, and then send the JSON back. Do it yourself? Yeah. Do it yourself. Speaker 1: You have more parameters available that way. Do we have is there sound up here? You can try. You can try? Alright. Speaker 0: You'll be the first. Speaker 1: Oh. Alright. I'm gonna skip ahead a little bit on this 1 Sai we're not here too long. Music output. This is, May Meetup 1.5, XL which Sai just released, like, last week. Speaker 1: Sai, yeah, album art is also generated on the server. Speaker 0: Are we birds? Fish? I don't even know. You use software development? Same thing, different bricks. Speaker 1: Yeah. So you can actually join the Discord instance To play with MuseBot yourself if you want to that's the QR code to join that. I'm also going to show you what MuseBot looks like live. I've got my friend helping me out with some some demos. Time gonna get everybody a moment to capture that if they wanted to. Speaker 1: Alright. And then, yeah, let's show you what Musebot looks like. To these are some of the image generation things. Now this is at an at home AI server. It's all local hardware. Speaker 1: So, if you do play with MuseBot, be patient. To. You're queued. You you are queued. I think probably 1 of the most interesting aspects of MuseBot is, the internal task queue system that I wrote. Speaker 1: I didn't wanna use something external, because I wanted to keep it, single binary as much as possible. So there's a task queue with multiple channels. Tasks in separate channels can run-in parallel. Tasks in the same channel have to be awaited by each other. Musebot can run-in parallel or serial mode. Speaker 1: Parallel it can utilize separate discrete graphics cards across COMPUI and OLAMA which is the LLC back end. Or there's serial mode where all tasks have to To each other unless they're a child task. Any questions? Yeah. What kind of hardware is this Speaker 0: running on? Speaker 1: So I have, so my friend over there actually donated the original hardware but today it is a Ryzen 7 2700X, 64 gigs of DDR4 memory, an RTX 3,060 with 12 gigs of eRAM, and an RTX 5,060 Ti with 16 gigs of eRAM suspended from the top of the case with zip ties because I ran out of room. Yes. Speaker 0: I'm curious how long like things like audio and then video, I'm sorry video generation are taking on the system now for your own? What are you gonna do in the future? Speaker 1: Sai, SDXL based image models run-in about 30 seconds. Flux based image models take about a minute. I have, there's a bot called Goliath which uses Quinn Image and Flux 2. That takes I think I've got that down to around 4 or 5 minutes. Video generation used to take over an hour, but with newer models and better hardware I've got that down to 5 minutes and that also now includes audio as you saw earlier. Speaker 1: And that's just that's for, it's about 1 minute per second of video. Alright. If there are no other questions, I will yield the floor. Speaker 0: On the Oh. Queuing, so that's what he was talking about. Like, that's what's queuing you up when you send a message and waiting for the hardware? Speaker 1: Yep. Speaker 0: Okay. Like, are there any issues that you've seen with, like, the queuing or the speed in terms of, like, waiting? Speaker 1: So, I've been experimenting more with serial queuing because I would like To to be able to use both GPUs, which is not the case right now. I'm still working out some kinks in the serial queueer, which also can do some really unpredictable results. I do have, some code in here that will, you know, it does flush COMPUI memory, it does flush Ollama memory through API calls. I've noticed that can be a little inconsistent or Time, because I have multiple Profit MuseBot instances, they all each have their own queue. I'm actually debating on making it so multiple instances can be hosted by 1, MuseBot binary. Speaker 1: Can you talk Speaker 0: to each other? Speaker 1: I have them so they're designed to ignore tags from each other, because that I did have an endless conversation going early on in the development Sai, and I didn't want that to become a possibility. Speaker 0: Thanks, Zachary. Yeah.