ai news

184 results

how deep does the rabbit hole go? try searching for "matrix"...

Show HN: CocoIndex – Open-Source Data framework for AI, built for data freshness

Hi HN, I’ve been working on CocoIndex, an open-source Data ETL framework to transform data for AI, optimized for data freshness.You can start a CocoIndex project with `pip install cocoindex` and declare a data flow that can build ETL like LEGO - build a RAG pipeline for vector embeddings, knowledge graphs, or extract, transform data with LLMs. It is a data processing framework beyond text. When you run the data flow either with live mode or batch mode, it will process the data incrementally with minimal recomputation and make it super fast to update the target stores on source changes.Get started video: <a href="https://www.youtube.com/watch?v=gv5R8nOXsWU" rel="nofollow">https://www.youtube.com/watch?v=gv5R8nOXsWU</a> Demo video: <a href="https://www.youtube.com/watch?v=ZnmyoHslBSc" rel="nofollow">https://www.youtube.com/watch?v=ZnmyoHslBSc</a>Previously, I’ve worked at Google on projects like search indexing and ETL infra for 8 years. After I left Google last year, I built various projects and went through pivoting hell. In all the projects I’ve built, data still sits in the center of the problem and I find myself focusing on building data infra other than the business logic I need for data transformation. The current prepackaged RAG-as-service doesn't serve my needs, because I need to choose a different strategy for the context, and I also need deduplication, clustering (items are related), and other custom features that are commonly needed. That’s where CocoIndex starts.A simple philosophy behind it - data transformation is similar to formulas in spreadsheets. The ground of truth is at the source data, and all the steps to transform, and final target store are derived data, and should be reactive based on the source change. If you use CocoIndex, you only need to worry about defining transformations like formulas.*Data flow paradigm* came in as an immediate choice - because there’s no side effect, lineage and observability just come out of the box.*Incremental processing* - If you are a data expert, an analogy would be a materialized view beyond SQL. The framework tracks pipeline states in database, and only reprocessing necessary portions. When data has changed, framework handles the change data capture comprehensively and combines the mechanism for push and pull. Then clear stale derived data/versions and re-index data based on tracking data/logic changes or data TTL settings. There’s lots of edge cases to do it right, for example, when a row is referenced in other places, and the row changes. These should be handled at the level of the framework.*At the compute engine level* - the framework should consider the multiple processes and concurrent updates. It should consider how to resume existing states from terminated execution. In the end, we want to build a framework that is easy to build with exceptional velocity, but scalable and robust in production.*Standardized the interface throughout the data flow* - really easy to plugin custom logic like LEGO; with a variety of native built-in components. One example is that it takes a few lines to switch among Qdrant, Postgres, Neo4j.CocoIndex is licensed under Apache 2.0 <a href="https://github.com/cocoindex-io/cocoindex">https://github.com/cocoindex-io/cocoindex</a>Getting started: <a href="https://cocoindex.io/docs/getting_started/quickstart" rel="nofollow">https://cocoindex.io/docs/getting_started/quickstart</a>Excited to learn your thoughts, and thank you so much! Linghua

Hacker News • Apr 23, 2025read more

Show HN: Morphik – Open-source RAG that understands PDF images, runs locally

Hey HN, we’re Adi and Arnav. A few months ago, we hit a wall trying to get LLMs to answer questions over research papers and instruction manuals. Everything worked fine, until the answer lived inside an image or diagram embedded in the PDF. Even GPT‑4o flubbed it (we recently tried O3 with the same, and surprisingly it flubbed it too). Naive RAG pipelines just pulled in some text chunks and ignored the rest.We took an invention disclosure PDF (<a href="https://drive.google.com/file/d/1ySzQgbNZkC5dPLtE3pnnVL2rW_9aTeuG/view?usp=sharing" rel="nofollow">https://drive.google.com/file/d/1ySzQgbNZkC5dPLtE3pnnVL2rW_9...</a>) containing an IRR‑vs‑frequency graph and asked GPT “From the graph, at what frequency is the IRR maximized?”. We originally tried this on gpt-4o, but while writing this used the new natively multimodal model o4‑mini‑high. After a 30‑second thinking pause, it asked for clarifications, then churned out buggy code, pulled data from the wrong page, and still couldn’t answer the question. We wrote up the full story with screenshots here: <a href="https://docs.morphik.ai/blogs/gpt-vs-morphik-multimodal">https://docs.morphik.ai/blogs/gpt-vs-morphik-multimodal</a>.We got frustrated enough to try fixing it ourselves.We built Morphik to do multimodal retrieval over documents like PDFs, where images and diagrams matter as much as the text.To do this, we use Colpali-style embeddings, which treat each document page as an image and generate multi-vector representations. These embeddings capture layout, typography, and visual context, allowing retrieval to get a whole table or schematic, not just nearby tokens. Along with vector search, this could now retrieve exact pages with relevant diagrams and pass them as images to the LLM to get relevant answers. It’s able to answer the question with an 8B llama 3.1 vision running locally!Early pharma testers hit our system with queries like "Which EGFR inhibitors at 50 mg showed ≥ 30% tumor reduction?" We correctly returned the right tables and plots, but still hit a bottleneck, we weren’t able to join the dots across multiple reports. So we built a knowledge graph: we tag entities in both text and images, normalize synonyms (Erlotinib → EGFR inhibitor), infer relations (e.g. administered_at, yields_reduction), and stitch everything into a graph. Now a single query could traverse that graph across documents and surface a coherent, cross‑document answer along with the correct pages as images.To illustrate that, and just for fun, we built a graph of 100 Paul Graham’s essays here: <a href="https://pggraph.streamlit.app/" rel="nofollow">https://pggraph.streamlit.app/</a> You can search for various nodes, (eg. startup, sam altman, paul graham and see corresponding connections). In our system, we create graphs and store the relevant text chunks along with the entities, so on querying, we can extract the relevant entity, do a search on the graph and pull in the text chunks of all connected nodes, improving cross document queries.For longer or multi-turn queries, we added persistent KV caching, which stores intermediate key-value states from transformer attention layers. Instead of recomputing attention from scratch every time, we reuse prior layers, speeding up repeated queries and letting us handle much longer context windows.We’re open‑source under the MIT Expat license: <a href="https://github.com/morphik-org/morphik-core">https://github.com/morphik-org/morphik-core</a>Would love to hear your RAG horror stories, what worked, what didn’t and any feedback on Morphik. We’re here for it.

Hacker News • Apr 22, 2025read more

Show HN: I open-sourced my AI toy company that runs on ESP32 and OpenAI realtime

Hi HN! Last year the project I launched here got a lot of good feedback on creating speech to speech AI on the ESP32. Recently I revamped the whole stack, iterated on that feedback and made our project fully open-source—all of the client, hardware, firmware code.This Github repo turns an ESP32-S3 into a realtime AI speech companion using the OpenAI Realtime API, Arduino WebSockets, Deno Edge Functions, and a full-stack web interface. You can talk to your own custom AI character, and it responds instantly.I couldn't find a resource that helped set up a reliable, secure websocket (WSS) AI speech to speech service. While there are several useful Text-To-Speech (TTS) and Speech-To-Text (STT) repos out there, I believe none gets Speech-To-Speech right. OpenAI launched an embedded-repo late last year which sets up WebRTC with ESP-IDF. However, it's not beginner friendly and doesn't have a server side component for business logic.This repo is an attempt at solving the above pains and creating a great speech to speech experience on Arduino with Secure Websockets using Edge Servers (with Deno/Supabase Edge Functions) for fast global connectivity and low latency.

Hacker News • Apr 22, 2025read more

Show HN: Open Codex – OpenAI Codex CLI with open-source LLMs

Hey HN,I’ve built Open Codex, a fully local, open-source alternative to OpenAI’s Codex CLI.My initial plan was to fork their project and extend it. I even started doing that. But it turned out their code has several leaky abstractions, which made it hard to override core behavior cleanly. Shortly after, OpenAI introduced breaking changes. Maintaining my customizations on top became increasingly difficult.So I rewrote the whole thing from scratch using Python. My version is designed to support local LLMs.Right now, it only works with phi-4-mini (GGUF) via lmstudio-community/Phi-4-mini-instruct-GGUF, but I plan to support more models. Everything is structured to be extendable.At the moment I only support single-shot mode, but I intend to add interactive (chat mode), function calling, and more.You can install it using Homebrew:<pre><code> brew tap codingmoh/open-codex brew install open-codex </code></pre> It's also published on PyPI:<pre><code> pip install open-codex </code></pre> Source: <a href="https://github.com/codingmoh/open-codex">https://github.com/codingmoh/open-codex</a>