When Apple answers a query using its smaller on-device AI, it’ll do so with a latency of 0.6 milliseconds per prompt token. (AFP)
4 min read . 12:30 PM IST
- While sophisticated AI tools process queries on cloud servers that need an internet link, Apple will run some AI queries via a small language model loaded on iPhones to make AI usage private and quick. Privacy is good, but will low speed deter consumer adoption?
