Machine Learning on jeffcarp

ML Hyperpolyglot

Tue, 24 Mar 2026 00:00:00 +0000

In the course of my work I found myself looking up the specs of various TPUs and GPUs quite frequently. There wasn’t a good resource that combined the two in a common table with all the relevant stats, so I made one. It may not be useful to anyone else, but I’ve been using it at least once a day. Check it out here and let me know if you find it useful. :)

A Debugging Journey into XLA, TPUs, and JAX

Mon, 23 Feb 2026 00:00:00 +0000

I published an account of an interesting* yak shaving journey I went on to debug a custom Pallas kernel on TPU. I’ll probably inline the content here at some point.

*Maybe to me and two other people on Earth

Helpful Mindset Shifts for GPU Users Looking at TPUs

Thu, 29 Jan 2026 00:00:00 +0000

Disclaimer: The views and opinions expressed in this post are my own and do not necessarily reflect the official policy or position of my employer.

As someone who’s worked with TPUs for a while, I’ve seen a common pattern in people coming from the GPU world: they’ll take a look at a TPU chip’s specs, then look at a comparable GPU, and see numbers for the GPU chip that are obviously higher and scratch their head. Let’s look at TPU v5p (the training-optimized chip released in 2023) compared to the Nvidia H200 (the flagship AI GPU also released in 2023):

Notes from Adaline Applied at AGI House

Wed, 11 Jun 2025 00:00:00 +0000

Last week I attended an AI product summit at the beautiful AGI House on the peninsula. It was a fun day getting exposed to what’s going on the frontier of product and AI. Here are my semi-structured notes.

The thorniest issue in tool use is identity and authentication
- Authentication bugs are much worse with agents because they can move faster (e.g. in 15 minutes an agent can exfiltrate a lot more data than a human).
Memory for LLMS will become more critical. MCP-based memory will enable portability across providers.
Tool descriptions are important, treat them like system prompts. An example of a great tool description is sequential thinking
Current models fall apart above 5 loaded tools
Current pattern is to delegate tasks to a sub-agent that’s very well tested against the specific inputs/outputs
MoE points to further specialization in agent space
“I’d rather have an agent that does one thing very well.”
“Tiny teams”
- Can you have super leverage by harnessing a fleet of agents?
Anthropic’s research into how long can a model complete a task correctly? Currently O(mins) - next step O(10s of mins)
Proactive agents (I thought this was super compelling)
- Background thread monitoring your life, chimes in when it can be useful.
There are two types of challenges in AI product right now: ones that will become solved as ai gets smarter, and those that will stay hard
Meme: every tool came from the abuse of context
Different software prioritizes different things, like being able to iterate quickly, or better reasoning around the whole system
Focus on where the handoff points are between humans and AI
OpenAI is about to launch a platform with value exchange, same as FB did with their dev platform, which they opened up then two years later rolled back and slurped up the popular use cases into their platform - Brian Balfour
- The game is to join it, get the distribution, then get off it and take your users with you ASAP
Brand is associated with a person at the company having an opinion in public in a specific space
- So: be out there and be visible in voicing your opinions as a thought leader and maybe iconoclast in a space
- It helps customers narrow down if their personality matches the brand
Communication is lossy - holding the entire product in one persons head is super valuable
User complaints don’t necessarily mean they’re switching
- Focus on what is the users “hell yes” case study
VCs are starting to focus on “quality of revenue”
You can build a moat with your good taste
- Being opinionated is a plus for founders

Cool tools mentioned

https://opentools.com
https://modelcontextprotocol.io/
https://www.ragie.ai/
Poe for ‘subscription fatigue’ :)
https://thesephist.com/posts/tools/
https://corecursive.com/024-software-as-a-reflection-of-values-with-bryan-cantrill/
https://www.granola.ai
https://decagon.ai/product/overview

How to Fake Multiple CPUs in JAX

Thu, 13 Mar 2025 00:00:00 +0000

Here’s how to emulate multiple CPUs when running JAX. This makes it easy to test multi-TPU/GPU code without actually needing the accelerators.

import os
os.environ['XLA_FLAGS'] = (
    os.environ.get('XLA_FLAGS', '') +
    " --xla_force_host_platform_device_count=8"
)

import jax
jax.devices()

[CpuDevice(id=0),
 CpuDevice(id=1),
 CpuDevice(id=2),
 CpuDevice(id=3),
 CpuDevice(id=4),
 CpuDevice(id=5),
 CpuDevice(id=6),
 CpuDevice(id=7)]

Doing Post-Quantum Cryptography in JAX

Sun, 05 Jan 2025 00:00:00 +0000

In 2018 while studying ML and cryptography around the same time, I realized that many cryptographic algorithms can be expressed as computation graphs, the same ones supported by major ML frameworks, which led to a completely frivolous attempt to implement cryptographic algorithms in TensorFlow, just to see if it would work.

The world has changed a lot since 2018. JAX is growing in popularity as an ML framework, and in the cryptography space, post-quantum cryptography has gone from a mostly theoretical threat to a slightly more real one. So continuing the motivation of the original post, I decided to look into implementing a post-quantum cryptography algorithm in JAX.

How I’m Learning JAX

Fri, 13 Dec 2024 00:00:00 +0000

Recently I’ve been trying to learn more about JAX, the next-gen ML framework from DeepMind. These are the resources I’ve found most helpful so far.

The JAX docs, which are excellent
Patrick Kidger’s Learning JAX as a PyTorch developer
JAX AI Stack tutorials
The book JAX in Action, along with accompanying Python notebooks
Codebases: Gemma, MaxText
How to Scale Your Model

If you know any more good pointers please let me know.

用20行Python构建Markov Chain语句生成器

Sat, 04 Jul 2020 10:44:41 -0800

A bot who can write a long letter with ease, cannot write ill.

—Jane Austen, Pride and Prejudice

这篇文章将引导您逐步学习如何使用Python从头开始编写马尔可夫链(Markov Chain)，以生成好像一个真实的人写的英语的全新句子。简·奥斯丁的《傲慢与偏见》(Pride and Prejudice by Jane Austen) 是我们用来构建马尔可夫链的文字。 Colab 上有一篇可运行的笔记本版本。

Read the English version of this post here.

Setup

首先下载“傲慢与偏见”的全文。

# 下载Pride and Prejudice和并切断头.
!curl https://www.gutenberg.org/files/1342/1342-0.txt | tail -n+32 > /content/pride-and-prejudice.txt

# 预览文件.
!head -n 10 /content/pride-and-prejudice.txt

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  707k  100  707k    0     0  1132k      0 --:--:-- --:--:-- --:--:-- 1130k
PRIDE AND PREJUDICE

By Jane Austen



Chapter 1


It is a truth universally acknowledged, that a single man in possession

添加一些必要的导入。

I'm Joining Waymo

Sun, 10 Nov 2019 15:38:38 -0800

Quick life update: I’ve left the Chrome team and joined Waymo (formerly the Google self-driving car project). I’ll be working on ML infrastructure.

It was a fantastic whirlwind 3 years working on infrastructure for Chromium and helping to–in a very small way–push the open web forward. On the team I launched wpt.fyi, a resource to help align the APIs of all browsers. I worked on syncing source code across repos. I launched a couple TensorFlow ML models. And I helped make the bug tracker quicker and more useful for everyone in the project.

How are Words Represented in Machine Learning?

Sat, 13 Jul 2019 01:31:29 +0000

Note:This post had a good run, but it is now very out of date. Check out newer content around tokenizers for a more up-to-date view.

Machine learning on human languages is a super exciting space right now. Applications are exploding—just think of how many natural language ML models it takes to run a smart assistant, from transforming spoken audio to text, to finding the exact part of a web page that answers your question, to choosing the correct words with the correct grammar to reply to you.

Build a Markov Chain Sentence Generator in 20 lines of Python

Wed, 16 Jan 2019 10:44:41 -0800

A bot who can write a long letter with ease, cannot write ill.

—Jane Austen, Pride and Prejudice

This post walks you through how to write a Markov Chain from scratch with Python in order to generate completely new sentences that resemble English.

The text we’ll be using to build the Markov Chain is Pride and Prejudice by Jane Austen. You can follow along here or grab a runnable notebook version of this post on Colab.

Doing Cryptography in TensorFlow

Sat, 23 Jun 2018 13:00:14 -0700

On the left: the Feistel Network from the DES cipher, implemented below. On the right: a deep neural network.

TensorFlow is a popular machine learning framework. If you look under the hood, TensorFlow is a general platform for doing computation over tensors in the structure of a graph.

How to Export Evaluation Results in Tensorflow

Fri, 05 Jan 2018 21:33:26 -0700

2019 update: just a heads up, this post is about TensorFlow 1.x. For TensorFlow 2.x, you probably want to check out Keras custom callbacks.

In TensorFlow if you’re using a tf.estimator model, for instance tf.estimator.DNNLinearCombinedClassifier, and as part of your automated training infrastructure you want to save the evaluation results as a JSON file, it’s not super straightforward, so here’s how to do it.

Let’s say you define your EvalSpec like this:

eval_spec = tf.estimator.EvalSpec(eval_input_fn,
  steps=hparams.eval_steps,
  exporters=[exporter],
  name='eval')

You’ll need to write a new exporter class that will take the eval_result from your evaluation step and save it to a file using the GFile API.

Example: Save and Load a TensorFlow Model

Sun, 19 Nov 2017 19:07:30 +0000

2020 update: just a heads up, this post is about TensorFlow 1.x. For TensorFlow 2.x, you probably want to check out this guide .

This post details how to save and load a TensorFlow model using the DNNClassifier API.

The key idea here is that you define a function or a class beforehand that takes a model directory (in which it will save and restore the model parameters), adds that to RunConfig, and returns a tf.contrib.learn.Estimator, for example, tf.contrib.learn.DNNClassifier. See make_estimator for more details.