Yeah it wasn’t great. Other parts of this week’s Weekend Update were better like the Meta AI character having the same dead eyes as Zuck or Tucker Carlson discussing movies. That wasn’t great but had some fun in pure ridiculousness.
Either way the fact this is even a sketch says a lot about where we are. They clearly think enough people know about this discussion, and him specifically, to be able to pull this off. If you made an SNL sketch about other people HN talks about the audience would just be totally confused and lost.
He has earned the very dubious honor of joining Elizabeth Holmes and Zuckerberg as “weirdo tech people to make fun of”.
For all the downsides of losing the business of government contractors, it really would hold Anthropic to their stated values: they’ll never develop AI controlled weapons on this blacklist.
Ok, if we're not being at all charitable with the language used by the hosts of the data, let's be equally uncharitable with OpenAI.
- If "OpenAI" means the company acting on behalf of the company, why were they even looking to do this?
- If "OpenAI" means they were acting as a proxy for bad actors, what actions do we take to handle that?
- If "OpenAI" means they were accidentally breaching this system, in what sense does that distinction even matter, in terms of the outcome? If I build a nuke by accident without eng. due diligence, am I legally liable?
Hell, we're really getting to the point where the damages that could be caused are like an arsonist in California on a 100F day with 100MPH winds. Who cares who's liable, they are going to burn half the damned state down and cause damage far in excess of their assets. If you don't want to suffer from it, you're going to have to find much better defense measures.
> If you don't want to suffer from it, you're going to have to find much better defense measures.
So if someone opens your unintentionally unlocked front door and steals your laptop, you don't care who's liable?
ETA: There are absolutely burn bans in place in the scenario you're talking about, and common sense prevents those from lighting fires otherwise. In the absolute extreme case that someone _ACCIDENTALLY_ set a fire, without negligence, we have a due process system to handle that. When I see evidence of this for the massive amounts of capital flowing into these companies, I'll gladly eat my words.
> So if someone opens your unintentionally unlocked front door and steals your laptop, you don't care who's liable?
There are literally hundreds of thousands of script kiddies poking around at servers all the time. Cloudflare stops 99.99% of them. You're still left with many entities from states to hobbists trying to crack your system after they've gotten past the CDN and captchas. If OpenAI got through, then somebody else could too. Somebody malicious even.
I appreciate when white hats inform companies, governments, and the public about their successful exploits. It helps keep the whole internet safer - even if just by waking up lax server admins.
> I appreciate when white hats inform companies, governments, and the public about their successful exploits.
Except it took 3 months for OpenAI to notice they did! OpenAI is not a white hat doing pentesting, but someone that is putting random materials on fire to see if they smell. Reckless, stupid, and criminal.
So what? OpenAI may be a problem, but the security of a government website rests on the administrators of that government website. Not on attackers playing nice.
> So if someone opens your unintentionally unlocked front door and steals your laptop, you don't care who's liable?
I'm not the person you're responding to, but...
If I hear my neighborhood has started to become targeted by people checking houses for unlocked doors and stealing stuff, I think an appropriate response for me is to ensure it is difficult or impossible for me to "inadvertently" leave my own doors unlocked so my stuff doesn't get stolen, and also to encourage or perhaps even enforce[1] my neighbors to ensure their doors are always locked to make this sort of theft impossible and remove the temptation for that sort of crime.
If you knew that there are 2 specific well known people in your neighborhood that are walking around trying to open doors and steal stuff - is your suggested solution for everyone to improve own security? I would say its far more economic to get the 2 people to stop or at the very least be more careful?
It's more like there are two locksmiths who try to open any door when someone pays them. Should we make them check if the person calling them is the home owner?
Yes, but it needs to be done the right way so legit customers don't get rejected, and you can't do things like make everyone mail them a photo of their ID because they could sell that to a thief or credit card scammer. Or someone could break into their office and take it. Or they sell it and pretend it was stolen.
And when you start talking about regulation the two locksmiths will say the best way to do it is ban lockpicking tools so they can keep them away from other less ethical people, or ban other locksmiths coming from the next town over.
If I found out this metaphor for neighborhood windows was actually a window into massive amounts of institutional data, then no, I don't want someone to talk me out of getting more security on my kitchen window.
Continuing with your metaphor - sure get more security - you likely can never eliminate a threat. But if the bad actors are reigned in at a systemic level, you can get away with a lock on your window instead of steel bars.
I'm of a view that in a society we are better off when we all can get away with lighter individual protections as otherwise, thos who cannot afford the necessary security end up suffering.
I clearly have committed the mortal sin of an imperfect analogy, which in this case may not even be relevant, as the article in question implies that the data wasn't left "unlocked."
But giving you the benefit of the doubt, what's the crime for actually breaking into a car that was left unlocked and taking things?
ETA: and furthermore, what crime is worse? And should it be?
He probably cares more about still having a laptop.
What are you getting at? Nobody’s saying that OpenAI aren’t or shouldn’t be liable for what their agents do. What I am implying above is that this is being blown out of proportion, especially since the article I’m seeing is about a politician saying things that he thinks will poll well with the anti-AI crowd.
Perhaps, yes, you could argue that "accessed" and "hacked" have a different intent.
However, Australia is showing itself to be one of the few countries to have a backbone against big tech. Still open to investment and setting policies towards new data centres, starting to debate copyright law reform, already implemented R16 social media bans.
"The AI agent encountered repeated blocks while seeking information from the government portal but found ways around them, ultimately gaining unauthorised access to other areas."
Your comment immediately gave more credence to OpenAI than him. I don't agree that's appropriate because OpenAI has a vested interest in that narrative and I attempted to challenge it in a way that doesn't default to OpenAI. The article in question is not from a publication I trust to be able to handle the technical details in a way that will resonate with their average audience.
> At a press conference in Sydney, Marles said the incident itself was “relatively minor” and that it appeared no personal health information had been accessed.
OK, the Chinese models have also hacked people. What exactly do you expect law enforcement to do.
While in the OAI case we can easily treat it as a law enforcement action. When Iran does it? What are you going to do start a war?
The forest in this analog is the internet. As you well know it is filled with threat actors that don't give two shits about your laws. You have been warned. It's your fault when you get burned and have exactly zero recourse.
In the OAI case where we can easily treat it as a law enforcement action, that's a far cry from "who cares who's liable." If the OAI case can be a law enforcement action, we're on the same page. The fact that sovereign nations don't land under our jurisdiction is not an argument against following up or restricting those that do. OAI is the only matter I'm commenting on.
Tying Sam to a post, calling him a witch, and burning some wood in the general vicinity for what OAI has pulled is a legitimate action.
The thing is this has zero effective power in stopping this ball that is all ready rolling. It's like the first time a buffer overflow was discovered and used illegally. If that person had been caught and been put in a meat grinder it has had zero effect on the exploits of future buffer overflows. A huge number of people mad at OAI (rightfully so, I want to be sure you understand that) think this will have any preventative effect for what is coming. It will not. A new era of risk is here. Worse if you just watched the the great orange idiot he's yelling full steam ahead, so expect very little to no action by the US government on this.
I simply find it completely absurd that instead of finding it great that these AI agents are finding security bugs for free, people are whining about banning the AI. What on earth is that even? What do they want then? Security by obscurity and pretending not to care about weaknesses?
I don't disagree with you - but I am curious as to the amount of power and effort that goes into something like this. I suspect that these hacks into systems are carried out by many agents, running on many MW of compute for a long time - how many actors have access to resources like that ?
In Australia we have some experience with this scenario (usually with higher temperatures) and some of the measures we have found effective are “total fire bans”, “jail the arsonists for extended periods of time”, and for negligent companies whose insufficient vigilance caused the fire specifically, “levy steep fines” and “find in favor of the plaintiff in class-action lawsuits for significant fractions of the company’s net worth”. Perhaps these measures could be of use here!
There using this framing to get the public used to the idea that LLMS can do all of this on there own without direct instruction. So criminals can hide there behavior behind agents.
If you dog bites the postman, you are liable, even if you didn't tell or encourage you dog to do it. You are responsible for your dog's actions.
You get to pay the postman's medical bills, you may get fined, and your dog may get put down - especially if this isn't the first time it's bitten someone.
OpenAI has "bitten the postman" many many times, and it's "owners" have boastewd about it and used it in their marketing.
How many more times should society allow this to happen before we say "enough" and hold Sam Altman and the board responsible and make them pay restitrution, and put it down?
If Albanese actually had a spine (as claimed elsewhere in this discussion), Sam and the board will be getting an invoice for all the time/expertise spend investigating this intrusion, and restitution for everybody who's
PII and PHI was exposed. (Although I suspect a good deal of responsibility for the data exposure rests with the people who designed and deployed the system that was breached. )
Yes, this is why all discussion about "safeguards" is maddening to me. They are simply committing crimes, and people should go to jail. I guarantee that OpenAI will stop worrying about "alignment" when people simply go to jail for hacking and theft.
Why the hell should we be charitable to companies who have no issue in looting everything in the public commons AND the pirate commons, for their exclusive benefit?
Or more pointed at OpenAI, "we're a nonprofit... LOL JUST KIDDING LOOT EVERYTHING!"
I agree with your sentiment, and no I’m not surprised, which is why I’m reading this as being “it was sitting on an unsecured S3 bucket but nobody was supposed to directly access it”.
> Prime Minister Anthony Albanese has revealed that an artificial intelligence agent developed by OpenAI infiltrated an Australian government website in June and accessed both public and non-public files.
That’s the first sentence, where’s this stuff about blocks and writing files?
I see what's happened here, you clicked the archive link that doesn't have the full article cached. Click on the OG link and you'll see it in its entirety.
The part about it writing files to the server suggests something more.
If not for that part, the rest of it does sound like a lot of weasel words. Why say “private files” instead of “not intended for public access”? The latter is confusingly unclear
> At a press conference in Sydney, Marles said the incident itself was “relatively minor” and that it appeared no personal health information had been accessed.
It's pretty clear something was left unsecured and the agent just "found" it
This is going to be something long the lines of someone coming in to your house after you left the door wide open. They should probably not have done that, any respectful person would not - but calling it a "breach" is really too much.
So this is not different than people who share Google drive docs with company data with public links. It's not a fault of OpenAI or any other company. With enough time even your laptop can do it. How do they 'know' OpenAI breached? Maybe someone had an agent running asking to do a scan on any public docs?
Exactly. Looking at more news coverage, it's very clear that the files that were "infiltrated" were publicly accessible. Why isn't the lack of basic data security the news story?
From itnews:
> While the portal is “public-facing”, according to Albanese, it appears not all of the data files that holds are for general consumption.
> “The AI agent accessed both public and non-public files,” Albanese said in comments broadcast by ABC News and other outlets.
> “The Medicare statistics reporting portal is a public-facing statistics portal that contains non-sensitive Medicare information relating to data and statistics such as spending.
It appears very much like, "Ok, sure, we put some stuff out in the open that we shouldn't have. But we had a robots.txt!!! Why didn't OpenAI respect that!?"
There's zero indication that any personal details were accessed, or even that any non-world-accessible data accessed, so this feels like a little bit of political spin has been added here.
My guess is that this incident was about to be detailed on OpenAI's new mea culpa list, and the Australian Government decided to ensure that nobody pointed fingers at them. Why make the news story about crappy government data security, when you can blame the nasty terminator bots instead?
Who, you may ask, would take that money? People like business influencer Megan Lieu, who chose not to disclose just how much she'd made from her AI deals, but says her biggest sponsorship to date has been with Anthropic (makers of Claude), as well as that her biggest sponsored contracts (for any client) are normally around the $30,000 mark.
That isn’t actually the point, the point is that the AI companies should be made to see that doing things for society is the only way they get any kudos.
If they want to compete to be seen as the good guy, by all means let them. But it means actually having to be the good guy, in at least some respects.
This is cheap. Plenty of scientists, many of whom are my friends, are working very hard on finding new therapies for cancer, they were doing it before genomic models came along and still doing it now. The amount of times something in the media is lauded as "holy grail" that is never heard from again because it either only works in mice or turns out to be toxic or 100s of different reasons is massive. In my opinion this attitude of putting rose glasses on is detrimental to scientific progress. People outside of cancer research routinely underestimate how hard it is to find a working protocol. I think it is better to have sober attitude because it allows one to see the limitations and challenges that need to be tackled, blindly hoping AI can solve everything and deliver miracle cures is exactly the attitude that lets people sit on their asses and do nothing.
Your comment is vacuous. There isn’t any other part of the picture. Trying to advance medicine is inherently good. It doesn’t matter how much you use it for marketing. None of what they have done is fake or useless, it is at worst early.
Misrepresenting matters of fact with regards to AI capabilities, their perceived threat to the public, as well as unorthodox accounting that wouldn't pass the smell test of a summer intern in Wall Street, all to fuel an unsustainable hype in the hopes that (1) the government enacts regulatory capture to create a moat for AI foundries by fiat and (2) the incoming IPO miraculously fixes the propped up valuations secured in private markets.
If that’s your picture then I see why you didn’t just come out and say it in your first reply. It’s all your own speculation and still has nothing to do with the fact that trying to make medical progress is a good thing.
Glad to hear how you feel about it. However, and this applies more generally than your comment above, what people online often forget is that there is such a thing as both positive and negative reinforcement. If you don’t reinforce good behaviours as well as denouncing bad ones, then you don’t get good outcomes.
> Anthropic’s head of influencer, Lexie Barnhorn, has described creators as essential to building trust in complicated technical products. Its strategy is partly consumer-to-business: People who adopt Claude personally may later introduce it in their workplaces.
> Anthropic’s best-known creator events have been smaller dinners and pop-ups in which Claude remained the ostensible subject.
The point I am making is if these companies are going to compete and spend more and more money for PR, then they should do it in a way that benefits society, which paying influencers does not do.
Let me know when OpenAI starts actively trying to cure diseases.
Personally I feel like Anthropic is underrepresented in "normie" marketing, all of my non-tech savy friends only know of ChatGPT and use "ChatGPT" in the same way my mom says "Nintendo" when talking about game consoles
I'm not going to say that it is absolutely Earth shattering (not that they claim that), but your comment is obviously wrong. In the paper they show experimental results where they express some of the proteins and show a phenotypic effect. They don't claim an exact function either, and are relatively restrained on the biology end of things. I fail to see how it is at the level of a vague shower thought.
> Although we don’t yet know its function, the system that Claude discovered has a set of characteristics that have only ever been found together in a handful of other systems, all of which are programmable and perform operations like cutting, copying, and pasting DNA. Beyond CRISPR, which has already transformed science and medicine, several other such systems are now in development as promising tools.
It's not novel, and they don't know if it means anything. They published it here for PR purposes.
This to me reads like an absolutely bog standard statement that would be made at any university PR piece about any novel nucleic acid system like this one. The paper itself will usually be more restrained but still attempt to gain some status via comparison.
It also is clearly novel in the scientific sense. This is not a known system and it may resemble some attributes of similar systems but it differs substantially in its arrangement, since it's not clearly a retron.
As for if it is for PR. Yes, I don't disagree about that.
Native transparency isn’t so hard to do by the way, I made an image AE (I don’t say VAE deliberately as none of these are VAEs, I don’t know why they keep being called that since the variational part is completely absent) that supported this about two years ago as a hobby project. I haven’t really been following the space recently, I’m surprised it’s taken so long for this to come out if it’s a first.
It’s not hard architecturally, but it is hard to find or create good datasets of images on the magnitude you want. I suspect the qwen team heavily used synthetic data for this.
Not very, I had a tiny dataset compared to a production run, I used a bunch of 3D renderings to augment the images with transparency. It worked pretty well given the scale.
It’s the message board(s) that the OpenAI agent swarm was able to communicate through via GET requests. I guess a bunch of vibe coded weekend projects based around this idea have now dropped.
reply