Aware that this is essentially a soon to be deprecated service. As any vendor you need to know the life expectancy of it, plus forecast/predict any pricing changes. You really do not want to become dependent. Even if competitors exist, there still some non trivial cost to switch
No this is the hard problem. HN people live in a bubble. Anything else than signup with oauth and then having access to everyone and its too hard and 99% of people will click away.
I think the friction is higher than that. With few exceptions I basically won't use any new service that requires me to create an account. I do all my online shopping on sites that offer guest checkout. Zero chance I'm creating any new social accounts at this point.
You using the word oauth means you're in the same bubble. People want signup with Google or Apple or even better, username/password signup or zero signup
> General Research Laboratories, LLC (“GRL”) is an aggregator of market research surveys in multiple marketplaces for business customers. GRL does not typically host consumer surveys which are conducted by other consumer-facing organizations....
So, just a middleman to give you fake/shady at best survey responses to pad your numbers, so big enterprises/concultancies can have data that say whatever they want. The rest of the website is just BS
Finally you get it. Only exception is that we run our own exchange now to do task bidding so we don’t need to deal exclusively with other companies to middleman. Core business is what’s called yield management (akin to DSP in adtech) where the best survey (is the user qualified for it, does it have the best pay, etc) is selected for traffic in <100ms. I’d only argue the shady companies are the ones paying proxies (like cint.com) and pushing paid user acquisition instead of surveys. We actively fund ontology development for better profiling targeting. but yes, big enterprises/consultancies can use and interpret the collected data however they want, not our responsibility and we have no legal rights over it anyway
Very fascinating, super interesting engineering. Although i do find it very funny how they just bypass a massive vulnerability, basically zero data isolation (even between good actors, let alone bad ones) with 3 sentences. Only in the llm space you can slap a massive limitation like this in the middle of the article and continue like nothing happened
> Whatever anyone tells Audel becomes part of the single experience that every other conversation draws on. In practice, Audel is bad at keeping secrets. Ask it what it’s been working on with someone else and it will often just tell you, even though we’ve asked it not to. We also haven’t studied what happens when two people give conflicting instructions. For now, we assume anything you tell Audel is shared with everyone on the team.
I'm curious, you say "super interesting engineering" but then they say "it will often just tell you, even though we’ve asked it not to" and to me that seems like extremely shit engineering.
Where are the interesting engineering parts at? Seems to be an interesting idea and perhaps design, but to call the implementation/engineering itself bad seems to be an understatement.
The security and the overall engineering were entirely separate items in that comment I think. It was explicitly called out that this is a security problem that you'd really only see treated in this manner in the LLM space. For what it's worth it's effectively unsolvable (AFAIU) short of realizing AGI with an amicable alignment.
What do you mean unsolvable? Don't give the LLM access to stuff it shouldn't, this is like Access Control 101, not sure how anyone can claim that particular problem is unsolvable?
Which again, pretty huge fuckup on OpenAI's side to run that sort of security testing on 3rd party hardware, and not on a airgapped machine, kind of amateur hour to be honest. Again, same principle, don't give it access to stuff you don't want it to access.
Should maybe have said more loudly in the posts that this is not intended to be disciplined engineering (or used in prod!)
we did think about the architecture quite a bit and we've tried a bunch of different things out. We are fully optimizing for velocity of research experimentation and iterations on the design vs engineering hygiene
i've been playing with the project and idea for years now. the codebase started as javascript plus supabase before reasoning models were a thing (i incorrectly predicted the world would call reasoning models Large Thinking Models - bahaha). at one point the focus was almost entirely on a really easy-to-use human prosemirror UI for editing the agent's thought stream--because the models were still so bad at reasoning at that time.
then we pivoted to all bash at some point since it simplified things and i have this hypothesis that the LLMs will do better if as much of its world as possible is command line operations (inspired by terminus the agent we built as part of the terminal bench project)
the bash implementation is entirely written by coding agents - the human time goes into iterating with an AI agent on the design docs in the ./design folder
re security model - i think of the project as a research experiment about what it's like to treat an agent more like a person - so for secrets you have to trust its judgement about what it will share or if it will betray your trust. I don't know the OpenClaw architecture in great depth but my sense is that none of the LLM based agents that can interact with multiple people have strong guards in place that prevent the agent from leaking secrets between people. Definitely an interesting area for research though!
so yeah security is not a priority in this research per se - easier to assume you don't share sensitive things with a headlong agent
Along those lines, because we keeping it research and optimizing the velocity with which we can try new things out, the current aim isn't necessarily to get a bunch of community contributions. but if we do start to get any i'm thinking we will actually require people to only create PRs with design docs (no code allowed) and maybe even also require the logs (or at least the human prompts and the agent's responses) from the coding session where they created the design doc.
maybe i should write up the history, philosophy, and big picture aims of the project at some point
The part they punt on ("we haven't studied what happens when two people give conflicting instructions") is the interesting part. That's not a memory problem, it's an authz problem. If everyone writes into one shared stream then there's no model of whose instructions bind the agent or who can override whom. It's resolving the conflict that will generate the greatest "learnings" and advance the agent. This basically becomes a tool designed to misbehave rather than a tool that will learn creatively.
We all know how conflicting instructions to AI end - "I'm sorry Dave. I'm afraid I can't do that"
Not all agents have access to a sandbox/cli/code execution environment to run arbitrary api calls etc. MCP helps by essentially having another tool call without needing a sandbox. If you do have a sandbox, then might as well do codemode if you insist on mcp https://blog.cloudflare.com/code-mode/
Devil's advocate will say "Well, the agent would need an MCP client to use MCP-served resources... if you can give it that, why not give it an HTTP client?"
The main difference from gateway is we help with picking the right voice stack, which seems to be a big problem for users: we benchmark the models continuously and route based on those measurements for your language and constraints, and the boards are public at https://benchmarks.speko.ai/
Second difference is where it runs. Our gateway is open source and runs in your own container, including with self-hosted livekit/pipecat. You get a temporary token before the session starts, and then your orchestration connects directly to the provider.
Vapi is a managed platform: you use their infra to use the voice AI stack. In our case you can have your own infra and switch between models, so you are not locked into a vendor. A lot of teams we talk to build their own infra as they mature, and that is where the router comes handy.
This is terrible. Models have been RLed on looking at the previous tool call chain, and reasoning. No chance this does not reduce performance. The point of compaction is that it also includes useful signal from the tool outputs itself so agent does not repeat it afterwards
Long context windows reduce performance and exceeding your context window is impossible. It should be a given that deleting context.. is just that. You do it when you must to preserve your session without compaction.
When a company gives away service a heavily subsidized service as a promo, the full cost of serving it (compute) can get classified as sales and marketing instead of just cost of revenue, which makes your gross margin look better!
> And gemma downloads also can be from auto CI pipelines etc. Nothing concrete
I have always found NPM download numbers truly suspect. Is no one caching? Are they estimating true number of downloads base on some estimate of cache hits?
> And gemma downloads also can be from auto CI pipelines etc. Nothing concrete
Thank you for adding some clarity to this. When I calculated 900 million downloads divided by 8.3 billion people in the world, I came with a number that made it look like about one person in 10 were downloading this model.
I said for a few years to many a downvote on HN, everyone wants AI, nobody wants to pay the true costs, the AI race will turn into a "race to the bottom" that is, who can give you the most compute for the lowest cost, and still remain profitable?
Personally, In SWE, i think the industry has made a grave mistake with the agents and we're just one big Catastrophe waiting to happen. I do think that there is very real value when software engineers use these tools as something akin to exoskeletons that allow the human to do more, rather than just fully replacing them. However, I'm finding more and more that companies are slop shops and just attempting to automate all of their software engineering. That will certainly end terribly. I hope we are not cannon fodder.
For what? There are some things I want AI for because it does it well. There are some things I don't want AI for because it just makes a mess (hallucinations). Maybe the next AI will be different and we will have the conversation again.
Indeed, it kills our planet, our culture and our economy extremely well. Oh yes, and some code monkeys enjoy that it can make computer code on the side.
> I said for a few years to many a downvote on HN, everyone wants AI, nobody wants to pay the true costs, the AI race will turn into a "race to the bottom" that is, who can give you the most compute for the lowest cost, and still remain profitable?
I keep seeing this but this line of thinking doesn't make any sense. What does it really mean?
There are expensive models that increase the probability of you doing your task under a lower cost. That means you can't use Gemma for coding your new compiler - it would just be overall costlier.
Heavier models are cheaper at more complicated tasks because they use fewer turns and fewer mistakes.
Cheaper models are more likely to be cheap at less complicated tasks. Like if you just ask Gemma "Hi" it would probably be cheaper than asking Opus.
So what does this statement really mean? People don't want to pay the extra for a more costly model? Why wouldn't you? It reduces your overall cost!
Because real Fable usage starts at $20/month, and has oppressive usage limits even at that (ridiculous) monthly price.
Compared to my $3/month GLM-5.2 subscription, I have never felt like I was leaving capabilities on the table by refusing to cough up $20 for 15 minutes of Fable use per day.
This is the wrong way to look at it. If you have a complicated task , you can solve it for cheaper if you used Fable. It will use fewer turns to achieve the same result.
You can solve it for cheaper if you use GLM but if you are involved in it more, but that defeats the purpose.
The point is that there aren't many complex tasks were fable delivers a significant value increase over cheaper models.
Single prompting a very complex tasks is rare even on frontier models, because it can be done successfully only for specific situations (e.g. you have a very strong verification step the model can iterate on).
Most of my everyday usage is for smaller takes, were you don't really get the benefit of the most expensive models, and my guess is that is the case for the most users
Again this is a resolution problem. Your tasks are small enough that fit into a nice $3 quota. If you are an enterprise or a power user, the right-sizing argument doesn't work.
I'm talking about API prices - subscription is a different game.
reply