AI vs Human Attention Spans

AI models were initially given human tests (like the LSAT or MCAT) or tests written for AIs like the MMLU. However since they’ve mastered so many of these tests and the tests don’t always carry over to real-world abilities, new measures of progress are needed. One way to rank the difficulty of a task is by how long it would take a human to complete it. In March, METR created a chart showing that AI is able to do increasingly longer software-related tasks:

In July they examined existing tests to create time estimates for range of subjects, and found a similar trend:

While AI is able to do longer and longer tasks, people’s attention spans may be declining. For leisure activities, there are many reports that people are no longer reading full-length books, they’re watching short video clips and even playing shorter video games. For workplace tasks, Gloria Mark gathered evidence that people stay focused on one screen for decreasing amounts of time:

(The chart is from her book Attention Span, image taken from this LessWrong post, see further discussion there.)

If it’s true that people’s attention spans are getting worse while AI’s “attention spans” are getting better, we have even less time left before AI surpasses us…