Show HN: Standardizing build, experiment, and deployment for LLM Development https://bit.ly/3L9xoBv

Show HN: Standardizing build, experiment, and deployment for LLM Development *Motivation* Hi hackers, I'm Asif. I know we dislike premature standardization, but hear me out. LLM Application development is extremely iterative, more so than most other types of application development. We need a process that allows us to iterate faster. LLM Development is highly iterative due to the activities that come with regular software development, as well as the need to make the LLM Application accurate and reduce hallucination. To improve hallucination, we need to trial and error various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, and possibly try multi-agent architecture. There are thousands of permutations to try, and we want to be able to easily experiment with different permutations and have a process to objectively judge LLM performance so we can iteratively move towards accuracy goals. *Solution* I have been working in AI since 2021 - first at FAANG with ML, then with LLM in start-ups since early 2023. I have had the chance to talk with many different companies that have been successful and unsuccessful with AI development. Using my learnings, I am working on an Open Source framework to standardize the build, experiment, and deploy process for LLM Development. The goal of this framework is to optimize for rapid iteration. We are doing this by enforcing a modular LLM application layer build, allowing for easy testing of different configurations of your application. We provide maximum flexibility for using any external tools you want for building your application. We also have tools to benchmark your accuracy and improve the performance of your application in a data-driven way. Finally, everything is deployable through a Docker image. *Getting Involved* If you're curious, check us out on Github. You can get fully set up with a single command. Stars for better visibility https://bit.ly/4eIjUKw July 1, 2024 at 02:51AM

Show HN: AI Rapper Online – Generate Personalized Rap Songs with AI https://bit.ly/4bspGNA

Show HN: AI Rapper Online – Generate Personalized Rap Songs with AI We’re excited to share AI Rapper Online, a revolutionary platform that allows you to create personalized rap songs using advanced AI technology. Whether you’re an aspiring rapper, a music enthusiast, or just curious about AI's potential in music creation, our platform offers a unique and fun experience. Features include: Rap Generator: Input themes, styles, and keywords to create unique rap songs. AI Rapper Online: Generate full rap albums with custom cover art. https://bit.ly/4byk58A June 30, 2024 at 04:56AM

Show HN: Let Grumpy AI Roast Your Startup Idea https://bit.ly/4eNJv4T

Show HN: Let Grumpy AI Roast Your Startup Idea https://bit.ly/3VDx2I8 June 30, 2024 at 10:22AM

Show HN: I built a create-your-own-story interactive novel that runs with GPT https://bit.ly/3zqdM9F

Show HN: I built a create-your-own-story interactive novel that runs with GPT I built this project around a year ago, but I didn't really publicize it a lot. I do play with it every now and then and figured I should share it with you guys because I enjoyed it and hopefully you will as well. https://bit.ly/3xu8zwZ June 30, 2024 at 05:30AM

Show HN: Taarr, Shell Script Manager https://bit.ly/4ck32s1

Show HN: Taarr, Shell Script Manager Shell script manager inspired by rerun and other tools. Can be used for runbooks, deploys, hooks, C2, configuration management, and builds. Several improvements and updated site. https://bit.ly/45NirP0 June 29, 2024 at 06:43AM

Show HN: Auto-generate a custom, developer-first playground for your LLM app https://bit.ly/3xDvANR

Show HN: Auto-generate a custom, developer-first playground for your LLM app Hi everyone - We’re the team at Inductor ( https://bit.ly/3W3HWbF ), where we’re building the tools that developers need to build production-ready LLM apps and product features – far more quickly, easily, and systematically. We’re super-excited to share a new capability that we’ve just built: Custom Playgrounds. We built Custom Playgrounds because it’s important to be able to interactively experiment with an LLM app or product feature as you’re building it, and share your work in order to collaborate – but, today, this is too often cumbersome to do. Custom Playgrounds solve this problem by providing a developer-first way to auto-generate and instantly share a playground running directly against your code and environment, with a single Inductor CLI command: $ pip install inductor $ inductor playground my.module:my_function where “my.module:my_function” is the fully qualified name of a Python function that is the entrypoint to your LLM app. (No modifications to your code required!) Some highlights: Custom Playgrounds enable you to Instantly auto-generate a custom playground UI that is customized to your LLM app or feature. Run your playground directly against your code and environment, and securely use private data sources or internal systems. Iterate faster, with an easy, powerful way to interactively experiment with your LLM app and then systematically evaluate it when you’re ready (e.g., with hot-reloading, auto-logging, test suites, and more). Collaborate with other people (e.g., other teammates, including those who are less technical) by securely sharing your work, collecting feedback, and leveraging their expertise (for prompt engineering and more). See our docs if you're interested in learning more: https://bit.ly/4clycPG We also have a demo video in our blog post: https://bit.ly/3W5qmUJ You can try Custom Playgrounds out for free by just running the `playground` CLI command above – you’ll be prompted to create an Inductor account (for free) if you don’t already have one. Alternatively, if you’d like to create your account first, you can sign up here: https://bit.ly/4cj0Ssz (you’ll receive further “Getting started” instructions after you sign in). We’d love to get any feedback that you have, and are very happy to answer any questions - drop us a note in the comments! June 29, 2024 at 12:41AM

Show HN: Thermion, an open source 3D rendering toolkit for Flutter and/or Dart https://bit.ly/4cmF3YX

Show HN: Thermion, an open source 3D rendering toolkit for Flutter and/or Dart I'd like to share Thermion, an open source 3D rendering toolkit I made for Flutter and/or Dart. This came about when I needed a 3D renderer for a Flutter app[0] I was building. At the time, the only off-the-shelf option was to embed Unity, which wasn't ideal for a few reasons - unstable build process, compositing and transparency issues, and licensing uncertainty, just to name a few. In any event, frankensteining Unity (C#) together with Flutter (Dart) didn't seem like a great solution. After browsing for alternative renderers, I discovered Filament[1], an open-source, cross-platform Physically Based Rendering engine from Google. I initially only wrote some simple bindings to embed Filament inside a Flutter Android/iOS app, but have since added more functionality, and have expanded this to cover Windows, MacOS and Web (there was Linux support at one point, too, but I haven't been keeping up with it I'm afraid). Last week I published the first Thermion release to pub.dev[2], and with that milestone it seemed a good time to share with HN. This is also a call for contributors if anyone wants to help[3]. Some examples: - A live DartPad playground so you can try the Dart-only (i.e. non-Flutter version) of Thermion in browser (Chrome only for now). https://bit.ly/3znZZjN - My personal website, where I added an interactive clone of myself https://bit.ly/3W5kgUl - Prouder (a Flutter app from one of the Thermion contributors) https://apple.co/4cqKNkz https://bit.ly/4coFp1w https://bit.ly/4clznOS Happy to share more details if anyone is interested! [0] https://bit.ly/3zomcya [1] https://bit.ly/4cBdNGB [2] https://bit.ly/3zkdiBP [3] https://bit.ly/3zo00nR https://bit.ly/4clsKMI June 28, 2024 at 09:47AM

Show HN: AI-Powered Alerts When Your Brand Gets Mentioned in Newsletters https://bit.ly/4ckGWWi

Show HN: AI-Powered Alerts When Your Brand Gets Mentioned in Newsletters Hey HackerNews community! Excited to introduce my latest project: a game-changer for staying on top of your brand's mentions in newsletters – introducing AI-Powered Real-Time Alerts! As a founder, I know the struggle of keeping tabs on where your brand appears. Our solution uses cutting-edge AI to monitor newsletters in real-time. Whether it's a mention in industry updates or a spotlight in subscriber emails, our technology ensures you're always in the know. Here’s what sets it apart: Real-Time Alerts: Instant notifications when your brand is mentioned. AI-Powered Monitoring: Advanced algorithms that scan newsletters efficiently. Customizable Criteria: Tailor alerts to specific keywords or contexts that matter to you. The tool not only saves time but also ensures you never miss a beat in your brand's narrative. Try it out and transform how you manage your brand presence in newsletters. Your feedback is invaluable – let's innovate together! Check it out and share your thoughts! https://bit.ly/4ckyLZQ June 28, 2024 at 09:40AM

Show HN: A new social giving platform https://bit.ly/4cnbdDO

Show HN: A new social giving platform Picture this: you’re scrolling through your social media feed, feeling the usual blend of amusement and boredom until suddenly, a video catches your eye. It’s not just entertaining; it’s transformative. It inspires you, educates you, or maybe just brings a smile to your face when you needed it most. You’re filled with gratitude for the creator behind this masterpiece, but a simple “like” or “share” feels inadequate. You want to do more, to express your appreciation in a meaningful way. That’s where Public Gives sweeps in, offering a platform where your gratitude translates into real generosity on a global scale, tangible support for the creators who enrich your life. Readmore: https://bit.ly/4ckk1Ku... https://bit.ly/3SatCMf June 28, 2024 at 09:15AM

Show HN: PodNarrator – Have articles read to you in your favorite podcast app https://bit.ly/3VLyBUw

Show HN: PodNarrator – Have articles read to you in your favorite podcast app https://bit.ly/3XLOJb5 June 28, 2024 at 07:03AM

Show HN: Siram – Automate Watering Your Houseplants https://bit.ly/3xMKDVv

Show HN: Siram – Automate Watering Your Houseplants https://bit.ly/3VB5VNG June 27, 2024 at 04:45PM

Show HN: Code to run Gemini (Nano) locally on desktop/Chrome https://bit.ly/45YSPip

Show HN: Code to run Gemini (Nano) locally on desktop/Chrome Chrome Canary (nightly build for devs) now has Gemini LLM inbuilt. This is just some simple code/demo to enable and use that feature. https://bit.ly/3LmlYdT June 26, 2024 at 08:01PM

Show HN: Standard Webhooks – simplifying 3rd party API's https://bit.ly/3VIsoZn

Show HN: Standard Webhooks – simplifying 3rd party API's https://bit.ly/4cCXpVp June 27, 2024 at 01:24AM

Show HN: I built an indie, browser-based MMORPG https://bit.ly/45GSYqk

Show HN: I built an indie, browser-based MMORPG I've been working on an MMORPG that is now in alpha as a solo developer. Here are the major open source technologies that I use: Blender - 3D modeling software for creating the overall environment and every game object. I've gotten a lot of CC and Public Domain assets from https://bit.ly/45JAdT3 GLTF - I export assets from blender to the GLTF asset format JSON - I write a JSON config for every game object that describes things like its name, its interactions, its collisions, etc. Node.js exporter - I iterate over the environment and every asset to create a scene hierarchy. I use gltf-transform for processing all GLTF files, compressing them, removing redundancies, etc. Node.js server - Uses express and socket.io to process game state updates. It keeps track of every client's game state and issues delta's at each game tick (currently 600ms). The client can send interactions with different objects. The server validates those and updates the game state accordingly. HTML/CSS/JavaScript/Three.js client - I use regular web technologies for the UI elements and three.js for the 3D rending on the browser. The client is responsible for rending the world state and providing the client with different interactions. All code is written in JavaScript which means less context switching. Performance seems to be good enough, and I figure I can always optimize the server code in C++ if necessary. I am currently running two cheap shared instances but based on my testing, they can likely support about 200 users each. This is a low-poly browser based game so it should be compatible across many devices. The data a user needs to download to play, including all 3d assets, is approximately 2 MB, even though there are hundreds of assets. Overall, it's been a fun project. Web development and open source software have progressed to the point that this is no longer an incredibly difficult feat. I feel like development is going pretty well and in a year or so there will be plenty of good content to play. https://bit.ly/4bkTATF June 26, 2024 at 06:53PM

Show HN: Voice bots with 500ms response times https://bit.ly/4bldFJB

Show HN: Voice bots with 500ms response times Last year when GPT-4 was released I started making lots of little voice + LLM experiments. Voice interfaces are fun; there are several interesting new problem spaces to explore. I'm convinced that voice is going to be a bigger and bigger part of how we all interact with generative AI. But one thing that's hard, today, is building voice bots that respond as quickly as humans do in conversation. A 500ms voice-to-voice response time is just barely possible with today's AI models. You can get down to 500ms if you: host transcription, LLM inference, and voice generation all together in one place; are careful about how you route and pipeline all the data; and the gods of both wifi and vram caching smile on you. Here's a demo of a 500ms-capable voice bot, plus a container you can deploy to run it yourself on an A10/A100/H100 if you want to: https://bit.ly/4bkHQAS We've been collecting lots of metrics. Here are typical numbers (in milliseconds) for all the easily measurable parts of the voice-to-voice response cycle. macOS mic input 40 opus encoding 30 network stack and transit 10 packet handling 2 jitter buffer 40 opus decoding 30 transcription and endpointing 200 llm ttfb 100 sentence aggregation 100 tts ttfb 80 opus encoding 30 packet handling 2 network stack and transit 10 jitter buffer 40 opus decoding 30 macOS speaker output 15 ---------------------------------- total ms 759 Everything in AI is changing all the time. LLMs with native audio input and output capabilities will likely make it easier to build fast-responding voice bots soon. But for the moment, I think this is the fastest possible approach/tech stack. https://bit.ly/4bkHQAS June 26, 2024 at 10:51PM

Show HN: Chrome extension that brings Claude Artifacts for ChatGPT https://bit.ly/3xK2RqC

Show HN: Chrome extension that brings Claude Artifacts for ChatGPT Hey HN, I've made a Chrome extension called Artifacts for ChatGPT, which brings the Claude Artifacts feature to the official ChatGPT website. https://bit.ly/3VxsenC June 26, 2024 at 02:57AM

Show HN: Best Sales Day Ever for My Side Project – Notion Branding Kit https://bit.ly/3VHCzgD

Show HN: Best Sales Day Ever for My Side Project – Notion Branding Kit Hi HN, I’m excited to share that my side project, Notion Branding Kit, just had its best sales day ever with 468 new orders in a single day! That’s a staggering 46700% increase! It's been an amazing journey building this project, and seeing such a tremendous response is incredibly fulfilling. I’m happy to share the strategies and insights that led to this success. If you’re interested in how to achieve big sales days for your projects, let’s chat! https://bit.ly/3VBIP9X June 25, 2024 at 10:05PM

Show HN: ControlFlow – open-source AI workflows https://bit.ly/3xFQvzO

Show HN: ControlFlow – open-source AI workflows Hi HN! I’m excited to share ControlFlow, our new open-source framework for building agentic workflows. ControlFlow is built around a core opinion that LLMs perform really well on small, well-defined tasks and run off the rails otherwise. I know that may seem obvious, but the key insight is that if you compose enough of these small tasks into a structured workflow, you can recover the kind of complex behaviors we associate with autonomous AIs, without sacrificing control or observability at each step. It ends up feeling a lot like writing a traditional software workflow. With ControlFlow you: - define structured tasks for your agents to achieve, - assign one or more specialized agents to each task, - then compose the tasks into a workflow for execution. We've been working toward this for a while, applying insights from our work on Prefect (our workflow engine) and Marvin (our LLM framework). ControlFlow is our effort to bridge the gap between all those impressive AI demos and what we see B2B users actually trying to accomplish in production. More often than not, their goal is to sanely delegate a few steps of a workflow to an AI, not deploy a suite of end-to-end autonomous while loops. This is ControlFlow's initial release and we’d love your feedback. We hope it's a refreshing take on making LLMs feel like software again! You can check out the repo here: https://bit.ly/4cylRr1 https://bit.ly/3VU10c9 June 26, 2024 at 12:16AM

Show HN: Singulatron – On-Premise AI Platform https://bit.ly/4clH9bS

Show HN: Singulatron – On-Premise AI Platform https://bit.ly/4catUKO June 25, 2024 at 01:11PM

Show HN: I made a remote job board with a free CV maker tool https://bit.ly/45LRn2I

Show HN: I made a remote job board with a free CV maker tool https://bit.ly/4cBT70s June 25, 2024 at 09:57AM