Because, as the gp pointed out, if the cost is least to the labs, then why not reap the benefits too?
Hypothetical. Assume you can in fact point agents at a tool and say "replicate it. Make no mistakes". You then have software being instantly copy-able.
Assume these agents can then be pointed to a customer feedback board in perpetuity and they autonomously upgrade the software over time. They analyze usage patterns and behave like PMs figuring out what to prune and what to build. Then the maintenance part of the stack also goes to zero.
Over time, the highest margin competitiveness will go to the distributor of the tokens. Aka the AI model makers.
In a world like that (which the frontier labs claim is within a year or two of happening) it feels like it's only a matter of time before they opt to own the entire stack down to the consumer apps. Kind of like Amazon deciding they want to knock off products doing well and then favour their own product over the original seller.
My guess is that if the capability arrives the only reason the frontier labs don't move to own the entire stack immediately is because of optics. Boil the frog instead.
There is more to selling software than writing it. You have market, support, and sell. Do you think their resources are well spent doing that across the gamut of software? Of course not; companies specialize.
Isn't the promise that LLM can do all this better than any human? Or at least in few short months? Surely marketing, support and selling is just case of right prompt?
I'm a little confused here. Cost of revenue is lower than revenue. That's good. R&D is the main contributor to losses here and this seems normal in an industry like this. For OpenAI specifically, I think this is problematic. They were the first movers but despite the large R&D they've lost so much ground to Anthropic despite Anthropic seemingly gifting them with weird PR self owns. But if we were to extrapolate this to the industry as a whole, this seems more positive than negative. Am I reading this incorrectly? Unless there's an assumption that R&D costs have to forever go up in order to increase revenue, I feel like this shows that the AI industry is actually on a path to profitability in the long term.
Whether it can physically be as all encompassing as it makes itself out to be or whether it will just be healthily profitable remains to be seen. Kind of like how Uber went from "We'll autonomously drive the world" to "Look, we deliver food, goods, and people to locations and we figured out how to do that in a way that makes profits. Also, ads".
I’m not sure how people are looking at numbers that show, even if we wipe off the enormous R&D expenditures, they are still in the red for inference + sales/marketing + admin and responding “this seems positive”.
It’s like being a sold a car and being told “well if you ignore the fact it has no engine it’s a good buy” yet it also has no wheels.
> Unless there's an assumption that R&D costs have to forever go up in order to increase revenue, I feel like this shows that the AI industry is actually on a path to profitability in the long term.
There are three futures right, I’ll rank them in order of fantasy -
1. Someone achieves AGI. At that point the economics of an individual company don’t even matter.
2. R&D costs do have to forever continue, because LLMs can be continually iteratively improved. Much like chip development, there is no end in sight, at least not on a near term timescale. If you are not continually at the frontier, customers will use a competitor or open/local alternatives.
3. LLMs reach a plateau of functionality. Further gains are minimal, quality reaches the apex of what the technology permits. In this scenario the hyperscalers have no business because open/local models will rapidly reach that same plateau as well.
The leaked numbers completely ignore how much of their compute is subsidized.
It also ignores how much of "R&D" is actually needed for the thing they offer to keep working. Looking at the thread everyone seems to be presuming "R&D" is all "training new models", but that is uncertain.
The people who are completely sold on the belief that AI providers are running at a profit believe him to be utterly, totally and completely wrong in every one of his predictions.
The people who are completely sold on the belief that AI providers are running at a loss they can never recover from believe him to be utterly, totally and completely correct in every one of his predictions.
The reality is that it's not his predictions that matter, but his data, which is almost always correct as of time of writing. If you ignore his opinions, the data presented on liabilities, spend, revenue, loans, commitments, etc across Coreweave, Stargate, Oracle and all of the usual AI companies is, as far as I can tell, correct.
IOW, when it comes to his opinions, it's all about your priors. His data is good, though.
> The reality is that it's not his predictions that matter, but his data, which is almost always correct as of time of writing. If you ignore his opinions, the data presented on liabilities, spend, revenue, loans, commitments, etc across Coreweave, Stargate, Oracle and all of the usual AI companies is, as far as I can tell, correct.
Yeah, I think that he does well with sources and data. I also think that his editorialising can be off-putting for lots of people. I kinda enjoy it, but accept that I have niche tastes.
Not only not understanding ARR, he simply doesn't do data analysis properly - he misses some few months and days in his calculation to prop up his point. This is a mistake chatgpt would have caught.
That's the trouble I have with ARR, because there's no standard, people engage in shenanigans. I do find the 5bn lifetime revenue versus their ARR figures pretty sketchy which is why I really want to see the S1.
Can you be more specific on his incorrect calculations please?
Wait. ARR has no precise definition but has a clear social understanding. It’s clear Ed doesn’t get that and ARR not having a clear definition doesn’t absolve him of the mistake. His misunderstanding was on a different axis.
The miscalculations are pretty clearly pointed out in the tweet I linked earlier.
> Wait. ARR has no precise definition but has a clear social understanding.
This is (historically) a recipe for fraud and badness. If ARR is important enough to be reported, then there should be a GAAP definition.
Do you use calendar month or four week rolling? Do you account for seasonality? How do you recognise revenue? (My sense is that Anthropic do sketchy things with credits, as the consumer ones last for like 180 days and then expire).
ARR is a really, really, really easy metric to make sound like whatever you want which is why I am sceptical of it.
EDIT: I looked at the tweet which is a screenshot of a supposed sheet that Ed built. Unless you have a source for the sheet then I'll need to assign this relatively low credibility (don't know the user, it's a screenshot with no link).
The user is someone I've followed for more than a decade:
> I’m a reporter who has written about technology, economics, and public policy for more than a decade. Before I launched Understanding AI, I wrote for the Washington Post, Vox.com, and Ars Technica. I have a master’s degree in computer science from Princeton.
> I’m working on Understanding AI full-time, and I have no outside investors or donors. Since I started it in 2023, paying subscribers have accounted for a large majority of my income (you can see full details on my source of income on my disclosure page). Their support allows me to work on the newsletter full-time.
Passes my credibility check because I've read a lot of his work, and he's been around the block a few times in journalism circles.
> But I’m a curious little critter and went ahead and added up all of the times that Anthropic had talked about its annualized revenue from 2025 onward, and the results — which you can find with links here! — and based on my calculations, just using published annualized revenues gets us to $4.837 billion.
It’s here in the blog.
> This is (historically) a recipe for fraud and badness. If ARR is important enough to be reported, then there should be a GAAP definition.
I don't think anyone believes the major AI providers are running at a profit? They are openly investing heavily into R&D and building out infrastructure, and according to these numbers way more than revenue. It wouldn't make sense for any of these companies to run at a profit right now as they're still aggressively expanding. The question is whether they will break even in the future, and capture a large enough market segment to sustain the business, allowing revenue to outgrow costs. If these numbers are real, revenue is already higher than COGS which is a really good signal for them.
I think the question is more about whether people believe this is a sound business in the long term, which imo isn't possible to tell based on these numbers yet.
> The people who are completely sold on the belief that AI providers are running at a loss they can never recover from believe him to be utterly, totally and completely correct in every one of his predictions.
It's funny, because you can both believe that these entities are bleeding money on every token and also believe that "financial engineering" will bail them out when they IPO despite this fact.
The fundamentals of running a business that sells products or services for more than the cost to produce them seem increasingly decoupled from the financial success of the company and its owners.
Up until this post, I thought he was someone with good financial insight, analytical chops, and business sense, stuck with an audience that thinks it's still 2023 and ChatGPT 3 is still the pinnacle of the technology, and that he therefore has to pander to in order to pay the bills.
After this supposedly being the reveal for his bubble-bursting massive revelation that will send the industry flying and lead to journalists kicking in his door for interview requests and exposés, I think... well, not that anymore. I thought "the frontier labs are losing money" was rather universally understood, and this really isn't even as bad as the stuff that's publicly visible; the fact that they keep raising hundreds of billions of dollars that they'll one day supposedly be required to show returns on?
> After this supposedly being the reveal for his bubble-bursting massive revelation that will send the industry flying and lead to journalists kicking in his door for interview requests and exposés
I mean, the fact that lots of expenses are not scaling with revenue (sales and marketing 5xed versus revenue 3xing) and that the losses are very very large is important. More importantly, these are audited figures which haven't been seen before.
Right, but this still isn't exactly new information. I don't think anyone was assuming that the labs are close to being profitable or that the losses wouldn't be rather large. The way this was announced was as if it was going to be a bombshell, but it just confirms what everyone (including the investors) was assuming anyway. Now if he had concrete numbers about whether inference at API pricing is profitable, that'd be a different thing (and it's what that hype bit was heavily implying since it's something he constantly keeps harping on, and rightfully so), but as it stands, nothing about these numbers says anything about whether this fundamentally has a road to profitability. It just says that this is a super high-risk high-reward investment, which isn't new information.
> As OpenAI’s worth rose, the increased value of those investor rights created a roughly $30bn charge, added the person. The charge is not expected to recur following the restructuring, they said.
> Stripping out the charge and other non-cash expenses, such as stock-based compensation of staff and computing credits from Microsoft, OpenAI’s losses were $8bn, according to the person.
As a long time FT subscriber, I'm happy you're using them as a source. The Zitron details were more useful to me though.
And none of my points have anything to do with the once off losses. I'm observing that a bunch of costs appear to be scaling with revenue or above revenue, which does not bode well for future profitability.
Also, as an aside, stripping out equity grants is really misleading for a private, high growth tech company.
The losses are scaling with revenue because increase in (expected) revenue increases valuation which increases compensation.
Once expectation stabilises these losses won’t happen because the valuation will remain constant.
A lot of people were paid really high equity grants simply because they started low. You can’t expect them to be paid the same amount each time.
FT themselves point this out and who you believe is up to you.
> The losses are scaling with revenue because increase in (expected) revenue increases valuation which increases compensation.
My original point around equity is that if you pay a substantial fraction of comp in this form, then leaving it out of expenses is pretty bizarre.
Is it your contention that the equity grants are the cause of their increasing losses?
I believe that this is probably not true at all, it's more likely to be S&M (salespeople scale as N not log(N) like engineering/product) particularly given that the product requires tuning for lots of companies (hence all the FDE hires).
More generally, the training costs seem to be increasing which is bad for their future profitability.
Also it should be obvious that you shouldn’t extrapolate stock based compensation in a scale up. People make a one time bounty but that is not recurring obviously.
> Before OpenAI’s switch late last year to become a public benefit corporation, investors in the company received convertible interest rights rather than conventional equity. Under US accounting rules, those interests were treated as liabilities and periodically revalued as the company’s valuation increased.
As OpenAI’s worth rose, the increased value of those investor rights created a roughly $30bn charge, added the person. The charge is not expected to recur following the restructuring, they said.
Stripping out the charge and other non-cash expenses, such as stock-based compensation of staff and computing credits from Microsoft, OpenAI’s losses were $8bn, according to the person.
I presume that this is what you're talking about, right?
That doesn't actually disagree with what I noted above using the (more detailed) figures from Ed's article. I noted that their revenue scaled by about 3x, while many costs (cost of revenue, sales & marketing, r&d) scaled by either equal (r&d) or greater than their revenue scaled. That's the point I was (apparently badly) making, nothing to do with the stock based compensation causing their losses. In any case, the loss was actually driven by treatment of the non-profit shares.
> Also it should be obvious that you shouldn’t extrapolate stock based compensation in a scale up. People make a one time bounty but that is not recurring obviously.
Correct, in some sense this is a once-off, however, most tech companies continue granting stock over time, so it's definitely worth including in actual margins. (This is a more general point that's not exclusive to Open AI).
The Uber comparison makes no sense. This is the opposite situation. Uber lost money on rides, OpenAI is (possibly) making money on inference. Uber used an R+D moonshot to autonomous driving to justify capturing an established industry without reducing costs meaningfully. OpenAI has a core product that risks becoming a commodity with open source models only 6 months behind.
Do you have a source for the claim that Uber was making money on rides during its decade of enormous unprofitability?
Its public stance was that growth was more important than profit. Why wouldn't they be subsidizing rides to fuel growth if that is their publicly stated goal?
And anyway, we got the Uber Files some years ago which made it explicit:
"In October 2014 in Madrid, the presentation shows, the hourly subsidy to drivers of $17.50 was almost twice the hourly fare it charged, which was only $9.10. In Berlin, the gross hourly fare Uber charged was $2.20, while the subsidy it paid out to drivers was $10.20 an hour. Uber burned through cash to “buy revenue”, in the words of the presentation."
The vast, vast amounts of money they spent on driver incentives city by city would seem to support the OPs claim (source: I was familiar with their spend on ads in the US approximately 10 years ago).
This is an impossible ask unless one works at Uber. I can tell you that i saw how much they were spending on ads back in 2016, and how long it continued and can assure you that they were 100% losing money back then.
Like, even now their margin is around 10% (they made 5bn on 50bn of revenue). Other software companies make a much, much, much better margin because Uber is basically not a real software business, it's an app attached to a low-margin delivery business.
Uber kept fares artificially low while simultaneously paying high bonuses to drivers to build a massive network. After burning through roughly $30+ billion over its first decade, Uber then pivoted its business model by raising rider fares, increasing restaurant fees on Uber Eats, and cutting driver pay.
"Cost of revenue" isn't the entire cost of running the company, (ie R&D, operations, sales, marketing, etc). It's just a cost they've associated with revenue IN ADDITION to the other costs I mentioned.
HSBC say they need to turn a 13b revenue to 200b by 2030 AND also find another 204b, in order to become profitable.
> It's just a cost they've associated with revenue
Its a little less arbitrary than that. Cost of Revenue/Cost of Sales/Cost of Goods Sold are clear, if you're following GAAP. To label these expenses as cost of revenue, they must meet the matching principle in that the expenses must be directly tied to the generation of specific revenue. If you didn't make that "sale" then that specific cost would not exist.
Other operating expenses come later on the income statement.
Total Revenue - Cost of Revenue = Gross Profit first, then you subtract OpEx from there for EBIT.
For OpenAI, I'd assume cost of revenue is almost directly inference costs + customer support & app dev.
How in the world could you read that article and think there is anything positive about OpenAI's prospects? We've been hearing for months that these companies need to make trillions of dollars in a handful of years, growing at record rates in order to break even and justify their massive outlay.
I tend not to focus on that future too much. I used to do so long ago. For example, how could Facebook possibly justify their losses while asking for such a big valuation? Same for Uber. Same for any number of big companies. And it turns out that growth in the future is impossible to predict accurately. Shopify is a good example where at the time the addressable market of online stores was tiny. But it turned out that Shopify created its own market which is huge today. Technology improvements have a way of creating new markets which far surpass today's total addressable market. Factor in currency depreciation and whatnot and sometimes, futures that looked impossible turn out to be possible.
Not saying anyone is wrong in pointing at the buildouts for AI and questioning its feasibility. Just making the argument for why I personally only look at operational costs and revenue because it's the only real-ish value I can look at and judge if a business can grow sustainably.
As a counter point, the red flag to all of this is R&D costs growing for each model release. If that continues and revenue cannot outstrip it, then these companies have a problem and it'll probably be that just 1-2 frontier labs can survive this once the dust settles.
I don't think Uber was a great ROI for investors though. It lost almost all the money they gave it in return for a business with entirely average profit margins (average across all industries, far lower than average for a SaaS app).
Since Uber's never paid dividends, ROI is easy to calculate.
At the end of its first day of trading (in May 2019), Uber's price was $41.57 per share, and it is currently $72–73 per share for a compound annual growth rate (CAGR) of roughly 8.2% per year.
In comparison, the BVP Nasdaq Emerging Cloud Index earned roughly 22–25% CAGR over the same time interval.
Tbh, there really needs to be some legal precedent set that makes model distillation a legal activity. If the model makers can rip everyone else's work and launder information as if it's their own without giving credit back to the original creators, I don't see why it should be illegal to distill the models. It's the same thing the frontier model makers are doing to IP everywhere else.
I agree. But this won't happen in the US because Anthropic / OpenAI is a big ol economic recession risk because we levered ourselves to the tits and put our chips on them.
OAI and Anthropic can actually both tank, MS would pick up OAI's IP, Amazon would pick up Anthropic's, and Google would keep cruising. We'd have a model plateau for a while but ultimate AI would keep on chugging.
If AI fails as a technology, it's going to lead to a great depression and probably either a revolution or WWIII.
If your country doesn't have any leading models, why not legalize distillation, either explicitly or implicitly?
(Chinese labs famously distilled American models, and that seems to be going well for them. They now have a competitive industry, home-grown talent choosing not to leave, and they now can truly compete without distillation).
Desperate to know what the prompt for the poem is. The idea of it felt familiar so I went down the rabbit hole and found: 14 years ago, a poem on reddit [https://www.reddit.com/r/RedditDayOf/comments/tjjw2/may_12_a...] . Nowhere near the length of the one the author shared but the same idea.
> This is from "The Cyberiad", a collection of science-fiction fairy tales by Polish author Stanislaw Lem ... In one of the stories, a robot constructor named Trurl creates a machine that writes poetry. A jealous rival named Klapaucian challenges the machine to compose "...a poem about a haircut! But lofty, noble, tragic, timeless, full of love, treachery, retribution, quiet heroism and in the face of certain doom! Six lines, cleverly rhymed, and every word beginning with the letter s!!"
And the computer responds with:
"Seduced, shaggy Samson snored.
She scissored short. Sorely shorn,
Soon shackled slave, Samson sighed.
Silently scheming,
Sightlessly seeking
Some savage, spectacular suicide"
The author had to be referencing this moment in their challenge to Fable/Mythos. I'm curious to know what their exact prompt was.
What's fascinating is that this is the difficulty of English translation -- which uses a different start letter and different words than the Polish one:
Right. But this is why I want to know the prompt. My hunch is that the author knew this story. But likely prompted Fable without hinting at it. And if so, the fact that Fable defaulted to the story of Samson shows that while it can impressively extend that over so many "scrolls", it also could only generate the idea based on what it had gobbled up. I'm thinking of this because given to another human, I doubt they'd only ever go for Samson's story by default.
That is a very poor comparison. Firstly, you're ignoring that behind the mass manufactured stuff, there's a lot of abused labour. People do protest that. Because of how money works, it's also impossible to avoid it.
Second, machinery that automated work isn't remotely the same. Engineers have built and refined the machines without having to go and inspect every new work that has been created by artisans each time. Creative people who have practiced the art of designing clothes and shoes stitch together and build prototypes. Entire machinery is built as an independent path away from how artisans build furniture.
There is a parallel though for how LLMs, in order to improve, gobble up all new work produced by people and never give attribution back. We see it when someone does a unique physical product design and starts selling it only for some 2 bit shop elsewhere to try and copy and sell a cheap knockoff version. The original person does all the hard work of prototyping and testing and the 2 bit shop which has access to more machinery resources buys a couple, copies it with less quality, makes a few changes, sells it, and probably outspends the original person on ad revenue too.
No, GenAI doesn't produce the exact same work as what they ingest. But style does get reproduced. And style is such a difficult problem to solve. Studio Ghibli didn't craft its signature style by accident. People prototyped and worked hard on how to design it, how to solve the problems unique to the design, created rules for it, and then painstakingly made the stories that were best told through that style. Only for the AI companies to pop out some bastardized version of it every time someone says "make my picture anime". No attribution given. No love. No homage. Just an encouragement for hordes of people to claim how easy it is without ever understanding the thought that actually went into it.
So no. It's not hypocrisy. It's recognition of these machines being information and creative laundering factories. They take and take and never give back any value that they could never create or improve on on their own. Those last words being key
The thesis of the person you're replying to seems to be that this is just another in the long line of mechanized crafts, and that it's hypocritical not to be equally anti-loom as anti LLM, for example. At least that's how I interpreted it.
While reading your rebuttal I was able to substitute LLM with loom and arrive at the same conclusions, mourning the loss of the artisan, copying their product for cheaper, etc. So you failed to draw the distinction that is necessary to rebut their point.
The only point you made that seemed on point to me was the first one, "it's not hypocrisy because people do protest looms", which I didn't find convincing.
I never thought I’d see the day when the open source “information wants to be free” crowd is complaining about intellectual property and “stylistic inspiration sources.”
The courts have already ruled on this. Creative work that is inspired by (ie. an amalgamation of) other works is not a copy. It’s how ALL creative work is generated by humans too. When enough humans get inspired/rip-off something we just legitimize it and call it a "genre."
The truth is most of the work done in the world is duplicative and not novel. LLMs are a giant compression model on that duplicative work, and if your job is to create charts and buttons out of react components for the 7,000,000th time, it turns out it’s better that AI automate that and free you up to focus on higher value tasks for humanity. Just as mass production eliminated duplicative artisanal work that made pretty much everything scarce and only available to the elite.
Did the rich elite lose some of the eccentric uniqueness in their world, and some of their previous performative signaling mechanisms (ie. putting 4 layers of hand carved crown molding in a ceiling differently than the last guy, or using $400k/yr engineers to create slightly different border radius buttons in each app)? Yes, but it comes at the benefit of the global masses who now can attain everything previously only available to elites via the AI mass production.
I loved visiting Versailles. Yet, I would never want to go back and live in that time, because there's a 99.99999% chance you aren't the guy living in it, and instead are the one who has 4 generations of your family enslaved to carve tiny sculptures into hand railings and live off gruel in a freezing cold hut while doing it. My lame, non-artisanal 2000s mass produced home is vastly superior to conditions that 99.99999% of people in the 1600s lived in.
> I never thought I’d see the day when the open source “information wants to be free” crowd is complaining about intellectual property and “stylistic inspiration sources.”
The difference is twofold: firstly people are intrinsically driven to be creative, that even with inspiration taken there's a desire to create something fresh. As you say, not just mechanical regurgitations of pre-revolutionary French style.
Secondly, "the courts have already ruled in this?" Have they? Are we not doing the IP thing anymore? Does that go for you and I and everyone else, or only for a handful of billion dollar companies?
There have been a million IP disputes over creative/artistic outputs going back hundreds of years. This is extremely well-tread territory. We don't need to re-invent IP law.
As much as people would like to completely own and charge rents on the idea of "Serif text on a black background with gradients," this is not a good outcome for society and we've already fought this battle and come to a good solution a long time ago. Creativity has flourished and is still flourishing because of it.
Do you think software patent trolls filing lawsuits on protected IP like "Phone application with informational dashboard" are good now because you hate and fear AI so much? Becoming pro-patent troll and pro-DRM would be a wild turn for the HN userbase, but seems to be what you're suggesting.
> Do you think software patent trolls filing lawsuits on protected IP like "Phone application with informational dashboard" are good now because you hate and fear AI so much? Becoming pro-patent troll and pro-DRM would be a wild turn for the HN userbase, but seems to be what you're suggesting.
Could you quote where I'm suggesting that? It seems like you're putting words in my mouth here and it makes discussion feel quite unpalatable.
> There have been a million IP disputes over creative/artistic outputs going back hundreds of years.
Yes and that's why I'm not allowed to torrent thousands of books to learn a particular style of painting or writing or whatever. I'm not allowed to scrape the whole Spotify library to help my music studies. But for these companies it's accepted practice. Odd
the debates is rich= bad, poor = good. The idea that they are all mediocre isn't popular on the left since it's a left dogma (and the reason why either the right always win or the left always loose).
Agreed. What's frustrating is that we have models for how sandboxing can work and instead of investing efforts into nailing that experience, the OS providers are prone to turning it into a monetization/lock in layer instead. My VLC and VS Code should have an OS native way of being limited to particular functionality. But when the OS providers implement the sandbox, they center it around an App Store and restrictions on only apps that have been notarized where said notorization costs money or a requires a subscription. And then they remove the ability to do things which their own native apps can do and set tighter controlling rules on what APIs apps can ever have access to.
When all I wanted was for VLC or similar to run in a sandbox by default where a plug-in I install can't do anything to my system or access the internet by default because the software itself is restricted to just the files I'm using and that's it.
That exists on linux under flatpak, but it requires Wayland and Pipewire. Also many packages just request full system permissions rather than update to work in a sandbox.
It's in the works and one day we will have it but progress is slow.
I really like openbsd's pledge. It's nice when you look at the code and see the program restrict itself to a smaller set of operations. Not everything in ports has adopted it, and the point is moot for closed source. But for the latter, VM and an isolated segment would be the proper solutions.
> My VLC and VS Code should have an OS native way of being limited to particular functionality.
The problem is... it's hard to scope. A media suite such as VLC, simply by what it is intended to do, needs a lot of permissions. Read data from physical media drives (CD/DVD/BD), preferably directly against the device to circumvent DRM. Access the network 0.0.0.0/0 1-65536 TCP and UDP to be able to play all sorts of streaming media. Access all files the user has access to on the computer because everything can be a media file and no operating system available does MIME type detection. Write to files on the user's computer to do stuff like format conversions and screen recordings. Access the screen framebuffer and the user's microphone for said screen recordings. Open network listen sockets to be a streaming endpoint.
Unless filesystems get a distinct metadata field to each file, there really is no viable way to sandbox it.
A viable strategy is something like qubeos for isolating activities from each other. You can have a media vm, a dev vm, a bank vm, and a password/manager vm. Or you use different computers.
I think one issues has been having code hosting/build systems/deployment pipelines under one ecosystem with non scoped keys. Especially your deployment keys should be on a service that only interacts with inert archive (no building or downloading anything).
I've been subscribed to ed for a long time. I commend his foundational ideas like what he laid out in "The Era of the Business Idiot" or "The Rot Economy". My recommendation line for him to anyone else is "if nothing else, he'll leave you with something to chew on for a while to come".
My issue with Ed is that he doesn't have the ability to draw the line. In the pursuit of making a point he goes so dogmatic that he is willing to make harsh statements that go beyond number backed predictions. Like in his piece "AI is really weird" he states about agents, "Probably the weirdest thing about this entire era is how nobody wants to talk about the fact that AI isn’t actually doing very much, and that AI agents are just chatbots plugged into an API.". That's a massive stretch to make. Just because he has a claim that the business doesn't make sense, he doesn't get to claim that agents are not capable of doing very real work. His assessment of cowork was "a chatbot that deleted every single one of a guy’s photos when he asked it to organize his wife’s desktop.". These statements damage his credibility and make it too easy to dismiss his writing as a rant of an angry man.
>"Probably the weirdest thing about this entire era is how nobody wants to talk about the fact that AI isn’t actually doing very much, and that AI agents are just chatbots plugged into an API." That's a massive stretch to make.
With the notable exception of TTI models, that description seems accurate to me. Is there any widely promoted "AI product" that is more than a chatbot in fancy dress?
Thats the thing though. The reduction of agents to Chatbots in a fancy dress doesn't make sense. Whether there's much of a moat to the models agentic approaches is a different question. But the idea of reframing questions and results in a back and forth between itself while holding on to context and all the laundered knowledge it has (no I'm never letting go of the lack of ethics in its knowledge acquisition) is an impressive feat. To say it isn't doing much is to liken someone doing any kind of thinking as doing nothing much. I'm not saying LLMs are sentient or intelligent in a human sense. But their synthetic intelligence does have capabilities that are impressive and they are capable of reducing so much busy work for me personally. Those ai agents that aren't apparently doing much can help me narrow down my reading about prior art in security implementations super quickly by going site by site, categorizing, locating the correct page in docs, or a Reddit discussion, extracting a relevant paragraph, sourcing it, and putting it down for me. The idea that this isn't much is reductive. That would have been 2 hours of my energy previously and instead I pick up the completed work, visit the links I asked the agent to get for me, do my reading in a pre planned structured way, and complete my work (I always try to be respectful of source material instead of attributing to the LLM). That is very useful behaviour and I think we thumb our nose at it to our own detriment.
Outside of coding agents, which are overhyped, I can't really think of any other agents that are particularly useful for getting work done. I installed cowork at one point and couldn't figure out anything I'd want to use it for. I guess there's maybe call center agents, which don't work great and live in an uncanny valley.
Personally I think the big benefits are gonna be in pretty specific business processes which are currently done manually. Lots of KYC and data entry between systems where it wasn't plausible to automate before are now doable.
I'm pretty uncertain around the businesses of foundation models but the tech is definitely useful.
Yep. I like to sometimes think of Agents as a slow but tireless unofficial API glue between completely disparate elements. I dislike the approach the companies took in laundering information through them. But their ability to work through clerical work that no one should be doing ideally has been incredible for me. Of course there's nuance to that last sentence. Someone needs to have an eye on certain things in domains like finance. But it's up to companies to be smart here and ask how AI can augment that process rather than trying to get rid of it. For example, collecting all relevant context and presenting it on demand for a suspicious transaction flag would eliminate what could be days of inter departmental wrangling in a normal work process.
My only side note of sadness here is that companies are more likely to implement such stuff in a haphazard way rather than anything actually thoughtful.
> My only side note of sadness here is that companies are more likely to implement such stuff in a haphazard way rather than anything actually thoughtful.
This will definitely happen, but the trend will end up towards automating a bunch of this stuff with manual checks on a subsample to ensure that it's working effectively. It's rather like moving from phone/mail based ordering to online ordering, in that it'll take a while but it's almost certainly gonna happen.
This is a misleading headline. It makes it seem like another supply chain attack where some good plug-in was taken over and used to deliver malware. Thats not the case here. Victims are invited to collaborate on a synced vault which comes preloaded with a non official plug-in that delivers the rat. Very very different story
The headline on HN is different: "Obsidian plugin was abused to deploy a remote access trojan". It's not a plugin that was abused, but the ability for shared vaults to contain plugins.
No. The attack does not depend on the presence of a specific plugin. The ones listed in the article are just the ones that were used in the POC. Any plugin could be modified by the attacker if the user trusts the attacker and accepts 1. the vault, 2. the shared plugins, 3. disables restricted mode.
Before software, there were accountants. It was The qualification to have.
Today accountants are still needed. But it's a commodified job. And you start at the absolute bottom of the bottom rungs and slave it out till you can separate yourself and take on a role on a path to CFO or some respectable level of seniority.
I'm oversimplifying here but that is sufficient to show A path forward for software engineers imo. In this parallel, most of us will become AI drivers. We'll go work in large companies but we'll also go work in a back room department of small to medium businesses, piloting AI on a bottom of the rung salary. Some folks will take on specialisms and gain certifications in difficult areas (similar to ACCA). Or maybe ultra competitive areas like how it is in actuarial science. Those few will eventually separate themselves and lead departments of software engineers (soon to be known as AI pilots). Others will embed in research and advance state of art that eventually is commoditized by AI. Those people will either be paid mega bucks or will be some poor academia based researcher.
The vast majority? Overworked drones having to be ready to stumble to their AI agent's interface when their boss calls them at 10 PM saying the directors want to see a feature setup for the meeting tomorrow.
Hypothetical. Assume you can in fact point agents at a tool and say "replicate it. Make no mistakes". You then have software being instantly copy-able.
Assume these agents can then be pointed to a customer feedback board in perpetuity and they autonomously upgrade the software over time. They analyze usage patterns and behave like PMs figuring out what to prune and what to build. Then the maintenance part of the stack also goes to zero.
Over time, the highest margin competitiveness will go to the distributor of the tokens. Aka the AI model makers.
In a world like that (which the frontier labs claim is within a year or two of happening) it feels like it's only a matter of time before they opt to own the entire stack down to the consumer apps. Kind of like Amazon deciding they want to knock off products doing well and then favour their own product over the original seller.
My guess is that if the capability arrives the only reason the frontier labs don't move to own the entire stack immediately is because of optics. Boil the frog instead.