• Echo Dot@feddit.uk
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 day ago

    Have any of the AI companies ever reduce their input output token prices I’ve always been under the impressive gone up. Not that I keep much of an eye on it.

    • boonhet@lemmy.zip
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 day ago

      The chinese models are getting cheaper now that they have “flash” variants

    • NuanceDemon@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      1
      arrow-down
      1
      ·
      1 day ago

      You’re talking about the cost to the consumer, where I was talking about the energy cost of an individual unit of work by a given model, but they’re both relevant. The underlying energy cost of generating a token at a given level of capability has been falling as hardware and inference become more efficient. Those efficiency gains can feed through into lower token prices for users, but also a more capable model can often complete the same task with fewer tokens, fewer retries and less prompting.

      So even if the headline price of a new frontier model looks similar or higher, the actual cost, both in energy and money, of getting a given piece of work done can still fall substantially. It’ll keep doing so too, its still in its infancy.