AI computational (inference) cost can be pushed to the edge of the network

Leave a comment
Artificial Intelligence

News today that Apple and Meta announced Muse Spark Open, which is essentially a frontier AI model (the jagged edge of AI capability) running on a local desktop computer.

As Mac Stories puts it:

Muse Spark Open is at the bleeding edge of models that can be run locally on computers that can sit on your desk and don’t need more power than can be delivered from a wall outlet.

The cost isn’t trivial, but not out of range of a small business either. Total cost of the hardware is going to comparable to a modern commercial-grade construction vehicle. (Pickup truck or Van). And owning or leasing this equipment would let you provide dedicated AI services for your small staff.

While the focus has been on the building of massive data centers, there’s a parallel revolution happening as well. I don’t think stopping data center construction will change things the way people hope. (And I’m not in favor of more data centers to be clear.)

The Author

Episcopal bishop, dad, astronomer, erstwhile dancer...

Leave a Reply