more parameters at low-precision is better than less parameters at high-precision

https://x.com/i/status/2039207501477097541

Reasoning ability comes from the structure of billions of connections, not the precision of each one. an 8b model with coarse weights still captures more of the original model's knowledge than a small model with precise weights.

The AI Race

Frontier intelligence in the West has become all about extrapolation and scaling laws. Meanwhile, those who are compute-constrained are left to gather the crumbs. I am incredibly grateful to Chinese AI companies for open-sourcing their models. It feels like the world has turned upside down in the tech race: the US used to lead in open-source tech, but now they have doubled down on proprietary solutions.

Geopolitics has divided the world of technology that once united us. Maybe it is time to adapt to the tech ecosystem created by our neighbor to the north... 

Life would have no meaning if not for its impermanence for every eventual permanence, there are many impermanence that build us fundamentally and I think that is beautiful.

C2PA should be mainstream

 As AI-generated images become the norm in society, it is imperative that we develop a method of provenance for certain media.

The C2PA (Coalition for Content Provenance and Authenticity) is one such system. It creates a secure, tamper-evident record of a piece of content’s origin and history. When a photo is taken by a camera, edited in software, or generated entirely by an AI model, C2PA can embed verified details like who or what created it, what tools were used, and whether it has been altered. This information travels with the file itself, protected by cryptography so it can’t be easily stripped or forged without detection.

One problem with this, though, is that it cannot prevent media that is screen-recorded. This just means that provenance is only as strong as the ecosystem that supports it. C2PA works best when platforms, devices, and creators ALL PARTICIPATE in preserving and honoring the chain of trust. And while it can’t stop every form of deception, it can establish a baseline of accountability for content that moves through compliant channels that people rely on, like the news and social media platforms.

Read More: C2PA | Verifying Media Content Sources