Last week-end, while the entire Tech world seemed focused on Silicon Valley Bank’s receivership, a number of interesting developments went relatively unnoticed. One breakthrough, one question and one lesson from the week end.
4-bit quantization on LLaMa, Meta’s Large Language Model
See Simon Willison’s story : Large language models are having their Stable Diffusion moment (simonwillison.net)
Large Language Models like ChatGPT are massive and, until last week-end, the consensus was that it would take years before anyone could run such large models on personal hardware. The fact that these models are somewhat strictly protected reduced the chances that “random” developers look under the hood and optimize them.
However, soon after the release of LLaMa, the whole model was leaked which kick-started a race to the best hack. It may be too early to declare a winner, but Georgi Gerganov stands out as a serious contender for the title: using the 4-bit quantization technique, the Bulgarian developer drastically reduced the model size, allowing virtually anyone with sufficient skills to run a GPT-competitive model on reasonably accessible hardware. Over the week-end, many developers gave it a try with convincing results, including LlaMa 7B (the “lighter” version) on Raspberry Pi 4. Someone even claimed to have run Llama 65B on a Mac (for reference, the “ChatGPT-equivalent” is LLaMA 13B).
That’s the very big deal of the week-end.
Samsung’s “space zoom” controversy
See Reddit post: Samsung "space zoom" moon shots are fake, and here is the proof : r/Android (reddit.com)
Ever since the “space zoom” feature was unveiled on Samsung’s new smartphone, there have been suspicions that, instead of heavily enhancing the pictures using AI, the feature would merely replace the picture by another one from a database. Huawei, before Samsung, had been accused of “faking” AI-enhancement capabilities using that trick.
Last week-end, a Reddit user posted pictures of the moon as proof that images were replaced. Bringing evidence based on pictures of the moon is not simple given the moon is tidally locked to earth (you always see the same side with the same craters…). The moon is not perfectly still though, judging on pictures available on the internet, so the case may be settled over the coming months.
Whatever the answer is, it begs the following question: at which point does a "real" picture edited with AI become an AI-generated picture inspired by reality?
Online banking makes everything easier, including bank runs
SVB’s failure at the end of last week wasn't a shock for its root cause, a liquidity mismatch between assets and liabilities (i.e banking 101 – though SVB wasn't helped by the fact that their clients burn more cash than typical depositors). Nor was it for the mountain of clueless comments on social media, at both extremes of the spectrum, during the week-end. But for how fast it went under. On Thursday last week, SVB’s depositors tried to move $42bn out of the bank, approximately 20% of all the bank’s assets, according to the regulator. Not a single bank could withstand that, even in jurisdictions where Asset-Liability-Matching (ALM - limiting the impact of interest rates variations) is a regulatory requirement and a liquidity buffer is mandatory. Banks just do not keep that much cash on balance sheet.
And that’s the lesson of the week-end: whatever the prudential regulations are, they are unlikely to prevent a bank run in the era of social media and online banking.
My two cents on SVB, as someone who once had a deep interest in financial crises and has tried to understand them over the years (it is also important for my job now) : the banking system is a critical piece of infrastructure, which people tend to underestimate these days. Most people have relatively fresh memories of 2008-2009 but Finance was the most fascinating at the end of the 1980’s as a largely unregulated sector following a decade of deregulation. One of my favorite stories from that time is that Apollo - now a global leading asset manager – initially started its business as a fund financed by a single investor, a French bank called Credit Lyonnais, and invested everything in a portfolio of junk bonds from failed Executive Life (acquiring Executive Life in the process). The transaction was hugely profitable – although it did not prevent Crédit Lyonnais from failing in 1993 - but US authorities came to consider that Crédit Lyonnais (a bank) effectively controlled Executive Life (an insurance company) which was illegal. After the Crédit Lyonnais bailout, the French government was prosecuted and held responsible. In a highly political context, it eventually paid a $750m-fine. Apollo was never a target of investigations and cooperated with the US authorities. If anything, Apollo thrived during the period.
But I got a bit carried away, you should read the full story here if you are interested. The bottom-line is that, almost every decade had major banking failures, and SVB had a comparatively strong asset-base : assets had lost value due to interest rates hikes so there was no doubt that shareholders were “out of the money”, but assets still had a value that was easy to assess. Potential buyers for the assets are already lining up (all those you have not heard on Twitter over the week-end as they were working so hard).
SVB is just another reminder in 2023 that banks can fail, and banks will fail (the collapse of North Channel Bank in Germany in January was an already serious wake-up call for the Finance industry). Regulators do not want or need to keep them all alive. Their role is to prevent a contagion to a systemic scale. Let’s see if they achieve that ; the whole plot may take months to unfold but the response to SVB’s failure over the week-end- protection of depositors but no bailout - was well received. The irony in that specific case is that many of those who placed their hopes on the fact that the regulator would handle the situation appropriately, had often been the most vocal about how incompetent or irrelevant regulators were.
