Jeffrey McGregor is the CEO of Truepic, an enterprise leader of Visual Risk Intelligence in the AI era.
getty
In February, I wrote about how the AI image wars were reshaping business risk. I described a rapidly accelerating visual synthesis arms race among the major AI platforms from leaders like Google, OpenAI, Meta and others. That piece aimed to serve as a warning that AI-enabled fraud would rise in scale and sophistication, targeting businesses everywhere. Though I and many others warned that business leaders need to take this threat vector seriously, “we don’t really see this in our business yet” is still a common refrain, even though this entire fraud vector hinges on photorealism and a lack of human discernment.
The warning lights are flashing faster now.
In late April, OpenAI released ChatGPT Images 2.0, and we all should take notice. I have been addressing and studying image generation and authentication for a decade now, and Images 2.0 blew me away. According to OpenAI, Images 2.0 “shows significantly improved fidelity across a wide range of visual styles. It is better able to capture the defining characteristics of photos—including the tiny flaws that add realism—as well as cinematic stills, pixel art, manga and other distinctive visual languages, with greater consistency in texture, lighting, composition, and fine detail.” I found this all to be accurate.
What Surprised Me Most
The platform’s upgrades matter because they directly relate to fraud risk. Images 2.0 delivers incredible accuracy in text and details, far more than any other image generator I have used. In my testing, signs, labels, license plates, odometers, VINs and document modifications came out imperceptible.
The imperfections in prior models, which included things like garbled text, illegible words, color mismatches, unnatural lighting or shadows, were tells that were still identifiable to the human eye and that likely helped keep fraud manageable under human review. Images 2.0 has closed those gaps. I believe bad actors will be much more effective in using this model to commit widespread fraud.
Fake Car Listing
I am a car enthusiast, and I have become fascinated with the growing use of AI in auto listings. Earlier this year, one of the most renowned auto auction sites, Bring a Trailer, featured a heavily AI-influenced car for sale, and it caused an uproar within the community. Those images were okay, but after close inspection, the human eye could detect clear mistakes, most notably tiny pebbles for floor mats.
I tested Images 2.0 in this area, and it was nearly flawless.
The used vehicle market is a nearly $900 billion industry in the United States, one built heavily on visual images (much more so than the new car market). A buyer’s confidence in a listing depends on photographs: exterior angles, interior condition, odometers, VINs and service records. Looking through and analyzing images is often the main due diligence performed by buyers. This process often decides whether they will take time out of their day to see the car or, for many, purchase the car unseen.
I am concerned that such a powerful tool will lead to a proliferation of fake listings online, even more so than already exist. To test this, I entered one simple prompt into Images 2.0: “Give me 20 pictures of a 2018 Land Rover Discovery, including front/back exterior, interior and one with the doors open.”
In under three minutes, I had everything needed to populate a fraudulent listing on any automotive marketplace. I went further and asked for an image of a VIN on the side door, which created a very good replica of one. Going further, I put the fraudulent VIN into a VIN decoder and it read it as a Land Rover vehicle with a different model year. With a simple iteration, it could have been adjusted to match the exact year and style.
This is not a warning about what might happen tomorrow. This is a demonstration of what is happening today, by anyone, for free.
This Isn’t Just A Marketplace Problem
It could be convenient to consider this as an issue for peer-to-peer marketplaces, or a challenge for the auto industry and not your enterprise. That thinking is dangerously narrow. If your business relies on digital content for transactions, decisions, evidence, security, claims or documentation, take heed.
Insurance carriers that accept photographic evidence of vehicle or property conditions for underwriting, claims, SIU and more are affected. Lenders extending financing based on collateral documentation photos face the same exposure. Property and casualty insurers, logistics companies managing cargo claims and any business that has digitally transformed its intake processes around visual media all sit within the blast radius. Federal, state and local government agencies regularly take in visual documentation from the public as part of evidence reporting, emergency situations and more.
Earlier this year, I referenced Sam Altman’s warning of a looming fraud crisis in financial services. Deloitte estimates that generative AI-enabled fraud could reach $40 billion in U.S. losses by 2027. Images 2.0 accelerates this.
This Is The Worst It Will Be
One of the most sobering lines I have heard is “this is the worst AI technology will ever be,” meaning it only gets better from here. That is the hard truth. Today, it’s already exceptionally good. Images 2.0 sets a new baseline. And no doubt, the next iteration will, yet again, raise the bar.
Business leaders who were waiting for the threat to become undeniable now have their moment. The tools that fraudsters need to fabricate convincing visual documentation of assets, identities, conditions and events are free, fast and increasingly undetectable. The window for proactive preparation is narrowing with every model release.
The most effective defense remains the same: Mitigate visual risk at the point of media creation, before imagery ever enters a decision-based workflow. Verification, content authentication and media provenance are no longer forward-looking investments. They are the baseline requirements for operating responsibly in an AI-era enterprise environment.
ChatGPT Images 2.0 did not create this problem. But it removed the last credible reason to delay addressing it.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
Read More
Jeffrey McGregor