Show HN: Using LLMs and Embeddings to classify application errors https://bit.ly/3PVG0yW

Show HN: Using LLMs and Embeddings to classify application errors Hi Hacker News! We’re Vadim and Chris from Highlight.io [1]. We do web app monitoring and are working on using LLMs/embeddings to add new functionality to our error monitoring product. Given that there’s a lot of founders/engineers using LLMs in their products, we figured we’d share how we built the new functionality, their impact on our workflows, and how you can try it out. Our goal was to build two features: (1) tagging errors (e.g. deeming an error as “authentication error” or a “database error”); and (2) grouping similar errors together (e.g. two errors that have a different stacktrace and body, but are semantically not very different). Each of these rely heavily on comparing text across our application. After some experimentation with the OpenAI embeddings API [3], we went ahead and hosted a private model instance of thenlper/gte-large (an open-source MIT licensed model), which is a 1024-dimension model running on an Intel Ice Lake 2 vCPU machine on Hugging face [4]. Our general approach for classifying/comparing text is as follows. As each set of tokens (i.e a string) comes in, our backend makes a request to an inference endpoint and receives a 1024-dimension float vector as a response (see the code here [5]). We then store that vector using pgvector [6]. To compare any two sets for similarity, we simply look at the Euclidian distance between their respective embeddings using the ivfflat index implemented by pgvector (example code here [7]). To tag errors, we assign an error its most relevant tag from a predetermined set decided by us. For example, if we tag an error as an "authentication error" or a "database error", we can allow developers to have a starting point before inspecting an issue.(see the logic here [8]). Anecdotally, this approach seems to work very well. For example, here are two authentication errors that got tagged as “Authentication Error”: * Firebase: A network AuthError has occurred * Error retrieving user from firebase api for email verification: cannot find user from uid. We also use these error embeddings to group similar errors. To decide whether an error joins a group or starts a new one, we decide on a distance threshold (using the euclidean distance) ahead of time. An interesting thing about this approach, compared to using a text-based heuristic, is that two errors with different stack traces can still be grouped together. Here’s an example: * github.com/highlight-run/highlight/backend/worker.(*Worker).ReportStripeUsage * github.com/highlight-run/highlight/backend/private-graph/graph.(*Resolver).GetSlackChannelsFromSlack.func1 Both reported as `integration api error` as they involve the Stripe and Slack integrations respectively. The neat thing is that the LLM can use the full context of an error and match based on the most relevant details about the error. We have rolled out a first version of the error grouping logic to our cloud product [9], and there’s a demo of all the functionality at [2]. Long-term, the HN community has other ideas of what we could build with LLM tooling in observability, we’re all ears. Let us know what you think! Links [1] https://bit.ly/46wquPv [2] https://bit.ly/46v63lZ [3] https://bit.ly/3XtPpzW [4] https://bit.ly/46cDDxd [5] https://bit.ly/46wypMF... [6] https://bit.ly/3PRZQek... [7] https://bit.ly/3tatjao... [8] https://bit.ly/46s8yoV... [9] https://bit.ly/3t7ySq1 https://bit.ly/465GcRv September 27, 2023 at 04:17PM

Show HN: Railway.app POC https://bit.ly/48v7K4w

Show HN: Railway.app POC https://bit.ly/48v7OkM September 27, 2023 at 05:33AM

Show HN: OnlineOrNot – Cron Job Monitoring https://bit.ly/3RCgoIp

Show HN: OnlineOrNot – Cron Job Monitoring https://bit.ly/3RAtPIQ September 27, 2023 at 11:45AM

Show HN: A JavaScript function that looks and behaves like a pipe operator https://bit.ly/3EUhOq8

Show HN: A JavaScript function that looks and behaves like a pipe operator https://bit.ly/48tGIdS September 27, 2023 at 08:31AM

Show HN: POC: Real-Time Medical Image Collaboration Made Easy with OHIF / Nun-Db https://bit.ly/3LETzjH

Show HN: POC: Real-Time Medical Image Collaboration Made Easy with OHIF / Nun-Db https://bit.ly/3ZxyWvj September 27, 2023 at 12:10AM

Show HN: XRain – Explore rainfall statistics around the world https://bit.ly/3RAaure

Show HN: XRain – Explore rainfall statistics around the world Last year I launched a website that allow people to see rainfall statistics that are based on satellite data. Historical rainfall information usually comes from rain gauges, and while these are fantastic there are many parts of the world that don't have many long term gauges, or where that data is hard to access. Satellite data can be an invaluable source of information for those data-scarce areas. The business model is to sell "extreme precipitation" data that can be used for flood modelling. Unfortunately, after a year I still haven't made a single sale. I've tried various ways of advertising, mainly via messaging people on LinkedIn who would actually have a use for it. I'm still proud of what I've built, even if it's a flop! Fun feature: jump to a completely random part of the world by clicking the "Random" button. I'd love feedback on anything, such as how to improve the UI/UX of the mobile view of the map page. https://bit.ly/45aviZs September 27, 2023 at 04:36AM

Show HN: Adding dynamic library loading to my pet programming language https://bit.ly/3ZuXqoW

Show HN: Adding dynamic library loading to my pet programming language https://bit.ly/3oEjfAP September 26, 2023 at 12:25PM

Show HN: Django ORM back end for RisingWave https://bit.ly/48FzhAu

Show HN: Django ORM back end for RisingWave https://bit.ly/48vy7re September 26, 2023 at 10:19AM

Show HN: Webcam Test https://bit.ly/463wUW9

Show HN: Webcam Test https://bit.ly/3tef5oN September 26, 2023 at 05:55AM

Show HN: Nature – Modern System-Level Programming Language and Compiler https://bit.ly/3LZJfTF

Show HN: Nature – Modern System-Level Programming Language and Compiler This is an introduction to the "Nature" project: https://bit.ly/3ET9TJS I am the author of the Nature language, and today I have released the 0.4.0-beta version. For detailed updates, please see https://bit.ly/3LZJhuL Now, Nature can easily interact with C language. It is anticipated that in about six months, Nature will release a version available to the community. By then, there will be stable syntax API, a complete standard library, and support for development tools. Moreover, all the standard library code in Nature will be merged into the main repository. Contributions are welcome. If you want to follow the development of Nature more closely, you can 'watch' it on GitHub to keep up with the progress. I also hope to receive your 'star' support, as it will give me great motivation. https://bit.ly/3rv8t59 September 26, 2023 at 01:21AM

Show HN: IntelliPlugin – The Future of WordPress Plugin Development https://bit.ly/3LCPlZA

Show HN: IntelliPlugin – The Future of WordPress Plugin Development https://bit.ly/3PA2TGH September 26, 2023 at 01:09AM

Show HN: A better command-line interface for Tailscale Taildrop https://bit.ly/46m2kqI

Show HN: A better command-line interface for Tailscale Taildrop https://bit.ly/3RwM3uL September 26, 2023 at 12:33AM

Show HN: Visualize your house in different styles https://bit.ly/3rqV51T

Show HN: Visualize your house in different styles I often find myself wanting to reimagine what a fresh coat of paint and some modern touches can do for houses in my neighborhood. This was a fun tool I made to solve that. Built with Replicate, Vercel, Next.js P.S. Still learning lots about how to best prompt the model to get best results. Would love to hear ideas/feedback. https://bit.ly/3rtX6uf September 25, 2023 at 06:54AM

Show HN: A central place to find custom ROMs for your Android device https://bit.ly/3rt3IJq

Show HN: A central place to find custom ROMs for your Android device https://bit.ly/3Ptxs0x September 25, 2023 at 08:38AM

Show HN: Animation Music Rose – JavaScript Web Art Tool on HTML5 Canvas https://bit.ly/46iYH5r

Show HN: Animation Music Rose – JavaScript Web Art Tool on HTML5 Canvas This program is created by CreateJS (JavaScript) + HTML5 Canvas. You can use it for your web pages and modify the design. https://bit.ly/3RzqV7g September 25, 2023 at 04:57AM

Show HN: Bots Playing Online Cash Bingo on an iPad https://bit.ly/3rsSDYQ

Show HN: Bots Playing Online Cash Bingo on an iPad Hey HN, I really enjoy playing "Bing King" from time to time. I had a little downtime, and thought it would be fun to try and write a bot to play it. Sharing it here: https://bit.ly/3LYktTV Demo videos in the repo https://bit.ly/3LYktTV September 25, 2023 at 01:53AM

Show HN: Emojirades – Emoji Charades https://bit.ly/3LCweyZ

Show HN: Emojirades – Emoji Charades https://bit.ly/460QWkr September 24, 2023 at 11:35PM

Show HN: Hichatbot.ai – Chat with your documents, video transcripts https://bit.ly/3PRfWF5

Show HN: Hichatbot.ai – Chat with your documents, video transcripts Hi HN, I created this site recently and need som feedbacks. Please give it a try, let me know if you see anything to need to be improved or added. Below is the summary of https://bit.ly/3RyAFi5 What is HiChatbot? - HiChatbot is an AI-powered chatbot that can answer your questions about documents, text, and videos. You can upload a document, text, or provide a video link to HiChatbot and have a Q&A chat with the content. What are some of the things I can do with HiChatbot? You can use HiChatbot to: - Summarize the main points of a document - Get more details about a specific topic in a document, text, or video - Based on the provided document, generate creative text formats of text content, like executive summaries, reports, poems, code, scripts, musical pieces, email, letters, etc. https://bit.ly/3RyAFi5 September 24, 2023 at 04:26PM

Show HN: pzip- blazing fast concurrent zip archiver and extractor https://bit.ly/3PPAptP

Show HN: pzip- blazing fast concurrent zip archiver and extractor https://bit.ly/3RAy1Zm September 24, 2023 at 06:23PM

Show HN: Krestomatio – managed e-learning platforms in public beta https://bit.ly/455ee7n

Show HN: Krestomatio – managed e-learning platforms in public beta Public beta available for our managed service that takes care of all the technical aspects of hosting open source Moodle™ e-learning platforms https://bit.ly/48pG5lv September 24, 2023 at 08:57AM