Lack of transparency in healthcare startups risks another Theranos implosion

Are more Theranos -style scandals looming for investors in healthcare startups? A team of researchers associated with the Meta Research Innovation Center at Stanford thinks so. They’ve  published a paper warning investors in life sciences startups that a systemic lack of transparency exists in their portfolio companies — creating the possibility for more multi-billion dollar implosions and […]

Are more Theranos -style scandals looming for investors in healthcare startups?

A team of researchers associated with the Meta Research Innovation Center at Stanford thinks so. They’ve  published a paper warning investors in life sciences startups that a systemic lack of transparency exists in their portfolio companies — creating the possibility for more multi-billion dollar implosions and scandals like the one that toppled Theranos and its charismatic founder, Elizabeth Holmes.

Indeed, one of the study’s authors, Dr. John Ioannidis, the co-director of the Meta-Research Innovation Center at Stanford and director of the University’s PhD program in Epidemiology and Clinical Research, was  among the first people to identify the risks associated with Theranos and its “stealth research”.

Now Dr. Ioannidis and his co-authors, Ioana A. Cristea and Eli M. Cahan have published a study surveying the publicly available research from the largest privately held companies in the healthcare space, and found them lacking. 

Most of the highest valued startups in healthcare have not published any significant scientific literature, the study found. Nearly half of the publications from companies worth over $1 billion came from only two startups — 23andMe and Adaptive Biotechnologies, according to the paper.

“Many years ago I was the first person to say that Theranos had a problem,” says Ioannidis. “The problem that I had then was that Theranos did not have any peer-reviewed evidence to show.”

In an interview and in their paper, Ioannidis and Cahan warn that investors have overlooked systemic problems created by the lack of transparency among healthcare startups by

They write:

“It would be tempting to dismiss the Theranos case as just one rotten apple. However, we worry that the focus on fraud puts aside a more fundamental concern. Fraud is making waves in the news, but stealth research may have a more detrimental impact.”

According to the study’s findings, more than half of the healthcare startups that are worth more than $1 billion have published no highly cited papers at all. For companies that were acquired or are publicly traded that number is around 40%.

In all, healthcare startups that are currently valued at over $1 billion published 425 Pubmed papers. And of those papers only 34 (8%, including 2 reviews) were highly cited. For companies with valuations of over $1 billion who had been acquired or are publicly traded on stock exchanges, the researchers counted 413 papers, of which 47 (11%, including 9 reviews) were highly-cited.

Digging deeper into some of the companies which had high valuations but little or no published research revealed scores of operational and technological issues for the researchers.

For instance, StemCentrx, which was bought for $10.2 billion in 2016 by AbbVie, had published 16 papers — and only one highly-cited paper. Since the acquisition, the Food and Drug Administration had imposed a delay on the readout of the company’s phase II trial for its Rova T targeted antibody drug for cancer treatment. In December a Phase III trial for Rova T as a second-line treatment for patients with advanced small cell lung cancer was halted, because the treatment wasn’t working, according to a report in Targeted Oncology

Acerta Pharma, another healthcare focused startup focused on cancer treatments was bought by AstraZeneca for $7.3 billion. That company published nine articles and had one highly-cited paper for a very early study of a potential treatment for relapsed chronic lymphocytic leukemia. Acerta received accelerated approval for a drug called acalabrutinib, which treats a rare form of lymphoma called mantle cell lymphoma. Two years ago, AstraZeneca had to retract data and admit that Acerta falsified preclinical data for its drug.

Then there’s Intarcia, the developer of a device for diabetes treatment that’s worth $5.5 billion. That company had its device rejected by the FDA and was forced to lay off staff and halt a couple of later stage trials. It had only published six papers — none of them very highly cited.

Ultimately, the researchers concluded that highly valued healthcare startups don’t contribute to published research and that the valuation of these companies by investors is divorced from any externally validated data.

For the researchers (and for investors) this should presents a problem.

“Many unicorns may be overvalued [21] and subject to unrealistic scientific expectations,” the study’s authors write. And they reject the argument that simply applying for — and receiving — patents is enough to prove that a technology in the healthcare space has been thoroughly vetted. “[Patents] do not offer the same level of documentation as peer-reviewed articles. For example, Theranos had over 100 patents [1], but these were unable to supplant the vacuum in their evidence,” the researchers wrote. 

Even if companies want to protect their technology, there are still ways for them to be more transparent about the results or benefits of their technology. The authors acknowledge that publishing isn’t the primary mission of startups. They can, however publish a few high-value articles, secure their technology through patents and then work with researchers, universities or hospitals to validate the technology and have those organizations publish results of the tests, the authors argue.

As the authors conclude:

Start-ups are key purveyors of innovation and disruption. Consequently, holding them to a minimal standard of evaluation from the scientific community is crucial. Participation in peer review, with all its limitations, is the best way we have to uphold this standard. We are not arguing that start-ups should divert excessive resources to having peer-reviewed papers. However, when their products are destined to affect patient health, they should neither be solely doing marketing. Confidential data sharing with potential investors or regulators cannot replace more open scrutiny by the scientific community.

 

Theranos documentary review: The Inventor’s horrifying optimism

A blood-splattered Theranos machine nearly pricks an employee struggling to fix it. This gruesome graphical rendering is what you’ll walk away from HBO’s “The Inventor” with. It finally gives a visual to the startup’s laboratory fraud detailed in words by John Carreyrou’s book “Bad Blood”. The documentary that premiered tonight at Sundance Film Festival explores […]

A blood-splattered Theranos machine nearly pricks an employee struggling to fix it. This gruesome graphical rendering is what you’ll walk away from HBO’s “The Inventor” with. It finally gives a visual to the startup’s laboratory fraud detailed in words by John Carreyrou’s book “Bad Blood”.

The documentary that premiered tonight at Sundance Film Festival explores how the move fast and break things ethos of Silicon Valley is “really dangerous when people’s lives are in the balance” as former employee and whistleblower Tyler Shultz says in the film. Theranos promised a medical testing device that made a single drop of blood from your finger more precise than a painful old-school syringe in your vein. What patients ended up using was so inaccurate it put their health in jeopardy.

But perhaps even more frightening is the willingness of Theranos CEO Elizabeth Holmes to delude herself and everyone around her in service of a seemingly benevolent mission. The documentary captures how good ideas can make people do bad things.

“The Inventor: Out For Blood In Silicon Valley” juxtaposes truthful interviews with the employees who eventually rebelled against Holmes with footage and media appearances of her blatantly lying to the world. It manages to stick to the emotion of the story rather than getting lost in the scientific discrepancies of Theranos’ deception.

The film opens and closes with close-ups of Holmes, demonstrating how the facts change her same gleaming smile and big blue eyes from the face of innovative potential to that of a sociopathic criminal. “I don’t have many secrets” she tells the camera at the start.

Though the film mentions early that her $9 billion-plus valuation company would wind up worth less than zero, it does a keen job of building empathy for her that it can tear down later. You see her tell sob stories of death in the family and repeat her line about building an end to having to say goodbye to loved ones too soon. You hear how she’s terrified of needles and how growing up, “my best friends were books.”

But then cracks start to emerge as old powerful men from professors to former cabinet members faun over Holmes and become enthralled in her cult of personality as validation snowballs. Oscar-winning director Alex Gibney has a knack for creeping dread from his experience making “Enron: The Smartest Guys In The Room” and “Going Clear: Scientology and the Prison of Belief.” He portrays Holmes’ delusions of grandeur with shots of her portrait beside those of Archimedes, Beethoven, and her idol Steve Jobs.

The first red flag comes when Holmes names her initial device Edison after the historic inventor the film assures you was quite a fraud himself. Soon, sources from inside the company relay how the Edison and subsequent Theranos hardware never worked right but that demos were faked for customers and investors. Instead of sticking to a firm timeline, Gibney bounces around to hammer home the emotional arcs of employees from excited to dubious, and of Holmes from confidence to paranoia.

Carreyrou’s “Bad Blood” meticulously chronicled every tiny warning sign that worried Theranos’ staff in order to build a case. But the author’s Wall Street Journal day job bled through, sapping the book of emotion and preventing it from seizing the grandeur of the tale’s climactic moments.

Gibney fills in the blanks with cringe-inducing scenes of Theranos’ faulty hardware. A ‘nanotainer’ of blood rolls off a table and fractures, a biohazard awaiting whoever tries to pick it up. The depiction of working in Theranos’ unregulated laboratory scored the biggest gasps from the Sundance audience. Former employees describe how Theranos recruited drifters they suspected of hepatitis as guinea pigs. Their stale blood evaporates into the air surrounding machines dripping with inky red, covered in broken test tubes. Gibney nails the graphics, zooming in on a needle spraying droplets as a robotic arm sputters through malfunctions. I almost had to look away as the film renders a hand reaching into the machine and only just dodging an erratic syringe.

A still from The Inventor: Out For Blood in Silicon Valley by Alex Gibney, an official selection of the Documentary Premieres program at the 2019 Sundance Film Festival. Courtesy of Sundance Institute | photo by Drew Kelly.

At times, Gibney goes a bit too melodramatic. The toy music box twinkling foreshadows a dream becoming a nightmare, but it gets maddening after an hour straight. The pacing feels uneven, sometimes bogged down in Holmes’ personal relationships when later it seems to speed through the company’s collapse.

Though elsewhere, the director harnesses the nervous laughter coping mechanism of the former employees to inject humor into the grim tale. With accuracy so low, Shultz jokes that “if people are testing themselves for syphilis with Theranos, there’s going to be a lot more syphilis in the world.” Visual dramatizations of journalists’ audio recordings of Holmes and the eventual legal disputes bring this evidence to life.

Alex Gibney, director of The Inventor: Out For Blood in Silicon Valley, an official selection of the Documentary Premieres program at the 2019 Sundance Film Festival. Courtesy of Sundance Institute.

The most touching scene sees Fortune’s Roger Parloff on the brink of implosion as he grapples with giving Holmes her first magazine cover story — momentum she used to eventually get Theranos’ useless hardware in front of real patients who depended on its results.

The Inventor succeeds at instilling the lesson without getting too preachy. It’s fine to be hopeful, but don’t ignore your concerns no matter how much you want something to be real. It takes an incredibly complex sequence of events and makes it at once gripping and informative. If you haven’t read “Bad Blood” or found it drab, “The Inventor” conveys the gravity of the debacle with a little more flare.

Yet the documentary also gives Holmes a bit too much benefit of the doubt, suggesting that hey, at least she was trying to do good in the world. In the after-film panel, Gibney said “She had a noble vision . . . I think that was part of why she was able to convince so many people and convince herself that what she was doing was great, which allowed her to lie so effectively.” Carreyrou followed up that “she was not intending to perpetrate a long con.”

Yet that’s easier to say for both the director and the author when neither of their works truly investigated the downstream health impacts of Theranos’ false positives and false negatives. If they’d tracked down people who delayed critical treatment or had their lives upended by the fear of a disease they didn’t have, I doubt Holmes would be cut so much slack.

Some degree of ‘Fake it ’til you make it’ might be essential to build hard technology startups. You must make people believe Inc something that doesn’t exist if you’re to pull in the funding and talent necessary to make it a reality. But it’s not just medical, hardware, or “atoms not bits” startups that must be allegiant to the truth. As Facebook and WhatsApps’ role in spreading misinformation that led to mob killings in India and Myanmar proved, having a grand mission doesn’t make you incapable of doing harm. A line must be drawn between optimism and dishonesty before it leads to drawing chalk outlines on the ground.

‘The Inventor,’ a documentary about Theranos and the ‘psychology of deception,’ will premiere at Sundance

Oscar-winning director Alex Gibney “will tell a Silicon Valley tale that [is] too good to be true,” in one of two films in the works documenting the Theranos saga.

A feature film chronicling the misadventures of Elizabeth Holmes, the criminally charged founder of the once high-flying and now-defunct biotech startup Theranos, will make its official debut at the Sundance Film Festival in Park City, Utah in 2019.

Titled “The Inventor: Out for Blood in Silicon Valley,” the film was directed by Alex Gibney and produced by Gibney, Jessie Deeter and Erin Edeiken. Gibney is an Oscar-winning director, known for his documentaries “Taxi to the Dark Side;” “Enron: The Smartest Guys in the Room;” and “Going Clear: Scientology and the Prison of Belief.”

Gibney teamed with HBO to investigate Holmes: “Drawing on extraordinary access to never-before-seen footage and testimony from key insiders, [Gibney] will tell a Silicon Valley tale that was too good to be true. With all the drama of a real-life heist film the … documentary will examine how this could have happened and who is responsible while exploring the psychology of deception,” HBO wrote of the project.

Holmes founded Theranos in 2003, dropping out of Stanford — as many tech luminaries have done — to disrupt healthcare. Her company garnered nearly $1 billion in venture capital funding from several high-profile investors and was at one point valued at north of $10 billion. She emerged as a celebrity in her own right and was touted as one of the youngest women to run a startup “unicorn.”

Then it all came crashing down.

Theranos’ claim to have invented blood tests that need just a single drop of blood was false. What followed were several lawsuits and a federal investigation that found Holmes and Theranos president Sunny Balwani guilty of “elaborate, years-long fraud in which they exaggerated or made false statements about the company’s technology, business, and financial performance.”

More details emerged when Wall Street Journal reporter John Carreyrou, who had closely followed the company’s rise and fall, published his book, “Bad Blood: Secrets and Lies in Silicon Valley,” earlier this year. A feature film for his account is in the works, too, which means we may get two Theranos movies in 2019.

“Bad Blood” the movie is expected to star Jennifer Lawrence as Holmes. “Shape of Water” writer Vanessa Taylor has signed on to write, with Oscar-nominated director Adam McKay (“The Big Short”) directing.

A record 14,259 films were submitted for approval to premiere at Sundance this year; just 112 features were chosen. Sundance runs from January 24 to February 3, 2019.

In Bad Blood, a pedestrian tale of heuristics and lies

In a world where thousands and thousands of startups are started in the Bay Area every year, becoming a name that everyone recognizes is no small feat. Theranos reached that summit, and it all came crashing down. The story of the fraudulent rise and precipitous fall of the company and its entrepreneur, Elizabeth Holmes, is […]

In a world where thousands and thousands of startups are started in the Bay Area every year, becoming a name that everyone recognizes is no small feat.

Theranos reached that summit, and it all came crashing down.

The story of the fraudulent rise and precipitous fall of the company and its entrepreneur, Elizabeth Holmes, is also the singular story of the journalist who chronicled the company. John Carreyrou’s tenacious and intrepid reporting at the Wall Street Journal would ultimately expose one of the largest frauds ever perpetrated in Silicon Valley.

Bad Blood is the culmination of that investigative reporting. The swift decline of Theranos and its protective legal apparatus has done this story a lot of good: many of the anonymous sources that underpinned Carreyrou’s WSJ coverage are now public and visible, allowing the author to weave together the various articles he published into a holistic and complete story.

And yet, what I found in the book was not all that thrilling or shocking, but rather astonishingly pedestrian.

Part of the challenge is Carreyrou’s laconic WSJ tone, with its “just the facts” attitude that is punctuated only occasionally by brief interludes on the motivations and psychology of its characters. That style is appreciated by this subscriber of the paper daily, but the book-length treatment suffers a bit from a lack of charisma.

The real challenge though is that the raw story — for all of its fraud — lacks the sort of verve that makes business thrillers like Barbarians at the Gate or Red Notice so engaging. The characters that Carreyrou has to work with just aren’t all that interesting. One could argue that perhaps the book is too early — with criminal charges filed and court trials coming, we may well learn much more about the conspiracy and its participants. But I don’t think so, mostly because the fraud seems so simple in its premise.

At the heart of this story is the use of heuristics by investors and customers to make their largest decisions. Theranos is a story of the snowball effect blown up to an avalanche: a retired and successful venture capitalist seeds the company, leading to other investors to see that name and invest, and onwards and upwards for more than a decade, eventually collecting a cast of characters around the table that includes James Mattis, the current Secretary of Defense, and Henry Kissinger.

Take Rupert Murdoch, the billionaire owner of News Corporation (and by extension the Wall Street Journal), who invested $125 million into Theranos near the end of the company’s story. He met Holmes at a dinner in Silicon Valley:

During the dinner, Holmes came over to Murdoch’s table, introduced herself, and chatted him up. The strong first impression she made on him was bolstered by [Yuri] Milner, who sang her praises when Murdoch later asked him what he thought of the young woman.

….

But unlike the big venture capital firms, he did no due diligence to speak of. The eighty-four-year-old mogul tended to just follow his gut, an approach that had served him well …

He made one call before investing $125 million.

To some readers, that might be a breathtaking sum, but it really is something of a pittance for Murdoch, whose reported net worth today is roughly $17 billion. In the denouement of the Theranos story, Carreyrou notes that, “The media mogul sold his stock back to Theranos for one dollar so he could claim a big tax write-off on his other earnings. With a fortune estimated at $12 billion, Murdoch could afford to lose more than $100 million on a bad investment.”

For Murdoch, a bad heuristic around the company cost him roughly 1% of his net wealth, and with the tax loss, may not have cost him much of anything at all.

That’s the challenge of the book: for all the fraud committed by Theranos and its founder, its financial losses were ultimately borne by the ultra-rich. This is not the 2008 Financial Crisis, where millions of people are thrown out of their homes due to the chicanery of Wall Street fat cats.

If there is a lesson in all of this, it is that the right heuristics would have helped these investors to an extraordinarily degree. Take for example the rapid turnover of Theranos’ workforce, which could have been checked on LinkedIn in minutes and would have signaled something deeply wrong with the company’s culture and leadership. It doesn’t take many questions to discover the fraud here if they are the right questions.

Beyond the investors and workers though, the harm is even hard to track to patients. There are perhaps no more serious consequences around Theranos’ fraud than for patients, who took tests on the company’s proprietary Edison machines and received inaccurate and at times faked results. Yet, Carreyrou strangely hasn’t compiled a compelling set of patients for whom Theranos caused morbidity. If any industry comes out positively in this book, it is the doctors of patients who reorder tests and ask additional questions when results didn’t make sense.

Ultimately, Bad Blood is a complete book about an important story. I’m reminded a bit of the 2012 documentary The Act of Killing, in which the filmmakers travel to Indonesia to have the killers of the 1965 communist genocide recreate the murders they perpetrated. The director’s cut is long and at times remarkably tedious, and yet, that is in many ways precisely the point. As a viewer, you become inured to the murder, bereft of emotion while waiting for the ending credits to roll.

Bad Blood is the same: its direct, to the point, and relatively sparing in any deep thrills. And that is its point. The book gives us a pinprick in our belief that Silicon Valley’s vaunted investors and founders are immune to stupidity. If you didn’t already know that before, you certainly now have a one-word household name of a startup to reference.