AI just got way faster

Groq Is AI’s New King of Speed

AI startup Groq (not to be confused with Elon’s Grok chatbot) went viral this week for its new chip that delivers near-instant chatbot response times. 

Some background: Groq is a processor (accessible via a cloud-based API) that can run open-source LLMs—like Llama 2 and Mixtral-8x7b—at incredible speeds.

• For comparison, ChatGPT-3.5 generates around 40 tokens (AKA words or numbers) per second…while Groq can generate upwards of 500 tokens per second.

Groq’s secret sauce? The company developed a new type of AI chip called a Language Processing Unit (LPU):

• Most of today’s AI models run on Graphic Processing Units (GPUs), which were originally built for computer graphics and video games.

• Meanwhile, LPUs are specifically designed for understanding and processing language-related tasks. They have fewer, but more specialized, processing units than GPUs.

Why does it matter?

Groq is introducing the next iteration of chips, built specifically for AI. If LPUs catch on, they could radically improve LLM speed and efficiency—potentially introducing a whole new world of LLM use cases.

Google Is Buying Your Reddit Posts

Better start polishing those memes! Your late-night Reddit rants and cat pictures might soon power the next big LLM.

Here's the scoop: On Thursday, Reddit struck a casual $60 million content licensing deal with Google—allowing Google to train its AI models on Reddit content through Reddit’s Data API.

• What’s in it for Reddit? For starters, a really well-timed revenue bump. On Wednesday, Reddit separately announced plans for its initial public offering (IPO)—and this deal certainly boosts its valuation. Plus, Reddit gets access to Google’s Vertex AI and cloud computing resources.

Why does it matter?

This move echoes OpenAI’s recent deal with media titan Axel Springer (the company behind Politico and Business Insider)—and it’s no coincidence. As content platforms lose out on advertising dollars to Tiktok and Meta, they’re looking for new revenue streams—and AI companies are ready to pay a premium for their data.

Google Goes Open-Source

Google has been making headlines with its proprietary AI models. But this week, the company dove head-first into the world of open-source AI. 

Introducing Gemma, a family of open-source models that enable developers to tap into some of Gemini’s capabilities without proprietary restrictions. 

In a nutshell: Gemma is built from the same research and technology used to train Google’s closed Gemini models. But there are some key differences:

• Gemma comes in two different model sizes, one trained on 2 billion parameters and one on 7 billion.

• Google claims that Gemma 7B outperforms its open-source competitors—Llama-2 and Mistral’s 7B models—on multiple performance benchmarks.

• While not as powerful as Gemini, Gemma models are small and efficient enough to run locally on a laptop—plus, they’re cheaper to run.

• Both Gemma models are available on HuggingFace, Kaggle, and Google’s Vertex AI platform. 

Why does it matter?

Google's pivot to open models raises questions about its broader AI strategy: Is this a calculated shift away from closed-source, or just Google’s attempt to compete with Meta and Mistral?

Is AI Controlling the Economy?

On Thursday, Nvidia shattered investors’ expectations with a record-breaking revenue announcement. 

• The company announced a record quarterly revenue of $22.1 billion for Q4 of 2024.

• That’s a 22% increase from the previous quarter, a 265% increase from the year before, and $2 billion greater than Wall Street’s predictions.

• This leap was primarily driven by its Data Center revenue, underscoring Nvidia's dominance and innovation in the AI chip industry. 

Why does it matter?

Nvidia’s unprecedented financial success represents the start of a massive market opportunity. As generative AI grows more ubiquitous, the demand for AI infrastructure will only grow. And as the third most valuable company on Wall Street (after Apple and Microsoft), Nvidia is ushering in the next era of tech giants.

• Stability AI launched Stable Diffusion 3, its most capable text-to-image model to date.

• OpenAI’s new Sora model will be coming to Microsoft Copilot. 

• Researchers just developed a revolutionary new type of AI chip that uses lightwaves instead of electricity.

• Adobe just rolled out a beta version of its AI Assistant to Acrobat. 

• ElevenLabs teasered a new AI tool that generates video sound effects from text prompts. 

• Match Group (the company behind Tinder) inked a deal with OpenAI.

More important AI news: Dive deeper into this week’s hottest AI news stories (because yes, there are even more) in my latest YouTube video:

Sora deep dive: Check out this video to learn everything you need to know about OpenAI’s groundbreaking new video model: