As far as I can tell, the paper says "bash capable", without ever describing what that means. How would one know whether a given model is "bash capable" or not?
I would have to imagine, that Luna would very much fall into the camp of "bash capable". At which point- it seems to me that adding any tools beyond just Bash requires some rigorous testing and verification that value is being added.
I think it’s sort of self-defined. If a model is able to use bash well enough to not need specific tools.
The research seems to agree with you, though. The paper calls out that for “bash capable” models, adding tools to do things bash can already do doesn’t improve performance.
Vaguely the same result as RAG. Unless you’re in specific domains, you won’t beat handing the agent a shell and grep.
> Vaguely the same result as RAG. Unless you’re in specific domains, you won’t beat handing the agent a shell and grep.
This has been my conclusion as well, and I'm doing my best to try and back this up quantitatively. In a perfect world, I could smite all of the internal MCP servers in my Corp environment and replace them with REST/GraphQL. No one using any of these servers is hitting them with models that would perform worse orchestrating with bash- and some of these folks are running harnesses with awful MCP clients.
I am still wondering about the effectiveness of using grep/awk (and ad-hoc python scripts) in code bases, as opposed to more sophisticated LSP and the like?
In many cases, the agent does hold the credential. When you authorize OpenClaw to read your gMail, OpenClaw has the credential. This is absolutely a poor practice, but common, nevertheless.
As for using the 'same tool', what I meant was that you the agent doesn't have to pick the tool at all. There is just one: the aclif CLI. Not separate tools for Salesforce, Docusign, Workday, etc that the agent needs to learn (and possibly mess up). Just the one aclif tool. Same grammar for all external services. Less agent inference the better.
Finally, alif CLIs support individual auth so a request can use SSO identities and fetch a token from a secrets value. The CLI holds the secret. If you deploy the CLI on a host or gateway, the agent never sees it.
Some really broad assumptions here, and youre being unclear.
"When you authorize OpenClaw to read your gMail, OpenClaw has the credential."
I can only assume you are implying that the execution environment accessible by the model via the harness here, had access to the credential. This is not even broadly true, as there are many single click solutions for deploying gateways that will allow operators to tls inspect and replace secrets in flight, outside of the agent execution environment.
Sure, not doing that is poor practice- but your phrasing is unfair.
"the agent doesn't have to pick the tool at all. There is just one: the aclif CLI"
Okay, so- if thats the only tool, then why are we even talking about credentials? Why are we talking about openclaw? The whole conversation regarding creds being in bad places is predicated on agents having native control over a sandbox- generally through shell. If your use case lets you bake whatever resource access is needed into a single tool- then many other security layers bubble up in value, being that you no longer need to authn/z arbitrary networked calls.
Also, yea sure- there is "one tool", all you've done is abstracted the tools into arguments.
"Less agent inference the better."
Show me the data, then. Show me how this performs better than the alternatives. This sounds like all you've done here is reinvent progressive disclosure?
"The CLI holds the secret. If you deploy the CLI on a host or gateway, the agent never sees it."
Okay, so- brokering, again. How are you solving authz then?
My response was simplified because your question was basic. OpenClaw was one example that fit the pattern, not a claim that every deployment holds the credential in the sandbox. Injection proxies exist and I should have said so. As for showing the data for advantages, I'll refer you to Anthropic and Cloudflare's Code Mode
We have our own, but will defer to 3rd party evidence.
Reinventing progressive disclosure is definitely part of this, but that's only one element of the approach. To be clear, this is all based on what we use internally, deployed in a particular way within our platform. Will leave it up to others to determine how useful it may be for them.
As for authz, that's what the rest of our system does and is beyond the scope of the aclif effort.
To say what the author said in another way- thats a garbage approach and completely misses the point. Sure, you can completely destroy the session by distilling it into a markdown- but thats like explaining what you did yesterday, vs having an immutable log of what you did yesterday.
Yea- its just a matter of whether the work is delegated to the client or offered by the server. Making sure what is served is in a really quality schema is generally the most efficient path- ime
Absolutely love openshell as a solution, I really hope Kube support moves out of experimental some time in the near future. Cool solution here- to a problem that I imagine is probably impossible to get 100%
Congrats on the launch here. I've been messing with this over the last few hours- super super cool. I was excitedly awaiting this hitting the API, because ofc there wasn't a super high fidelity option for drop-in voice interface in front of a given harness. This is blowing me away so far!
(Side q, is there a single place one can watch for updates on the API- that actually covers everything that changes? IIRC there have been a couple of additions that you've tweeted- but never hit the API changelog ;] )
And replying with some more thoughts after reading the comments here. To me, this feels similar to when models started passing the line of (imo) "good enough" to start building much more capable agents. The release of this (gpt-live-1) in the app felt like a big jump in capability, and now that this is available on the api- and I've tried it, it really feels like something big is unlocked for devs. Using this as the UI for a harness feels good so far, and its very much just plug and play. I would encourage others to throw their coding agent of choice at the docs- and have it spin up a web page that puts this infront of a capable harness; it feels 1:1 with the current voice mode in the OAI app(s), and you can define the tool surface yourself. Its really cool.
Congrats on the launch here. I've been messing with this over the last few hours- super super cool. I was excitedly awaiting this hitting the API, because ofc there wasn't a super high fidelity option for drop-in voice interface in front of a given harness. This is blowing me away so far!
(Side q, is there a single place one can watch for updates on the API- that actually covers everything that changes? IIRC there have been a couple of additions that you've tweeted- but never hit the API changelog ;] )
I would have to imagine, that Luna would very much fall into the camp of "bash capable". At which point- it seems to me that adding any tools beyond just Bash requires some rigorous testing and verification that value is being added.
reply