xAI has announced that its Grok chatbot is getting an upgrade, with improved reasoning, particularly on math-related tasks, expanded contextual logic, and overall performance enhancements.
xAI says that the latest version of Grok will bring it up to par with other chatbots on the market, and even exceed them on several benchmarks.
As per xAI:
“One of the most notable improvements in Grok-1.5 is its performance in coding and math-related tasks. In our tests, Grok-1.5 achieved a 50.6% score on the MATH benchmark and a 90% score on the GSM8K benchmark, two math benchmarks covering a wide range of grade school to high school competition problems. Additionally, it scored 74.1% on the HumanEval benchmark, which evaluates code generation and problem-solving abilities.”
So Grok should be able to provide better responses, faster, on various tasks. Though at the same time, I would love to see the data on just how many people are actually using Grok at present.
Originally released back in November, Grok is X’s answer to ChatGPT, with users able to pose questions to the bot and get generated answers.
What’s more, you can put Grok in “Fun Mode” and get more sassy responses, which Elon and Co. are convinced is a differentiating feature.
Well, that and Grok isn’t “woke”, which every other AI chatbot is, according to Musk, while Grok is also the only chatbot powered by real-time posts on X, which should give it an advantage in up-to-the-minute context, etc.
So, theoretically, Grok should be better than other chatbots on certain tasks, but then again, Grok doesn’t have the same compute power as ChatGPT or Gemini or Meta’s Llama-powered models.
So is Grok any good?
Well, we don’t really know, because thus far, Grok access has been limited to X Premium+ subscribers, of which, there are very few.
X Premium overall has fewer than a million subscribers, and that’s inclusive of all the people paying $8 and $3 per month for the regular and basic packages. Few are paying $16 per month for the top-tier Premium+, and as such, there are not a lot people that can even access the bot to share their experience.
X is looking to change this, by making Grok available to all Premium subscribers, while it’s also gifting Premium to highly followed users.
Ideally, that’ll kick off the Grok hype train, and maybe bring more people to the chatbot. But more usage will also highlight more errors and issues, which could also expose flaws with the Grok system.
Which we’ve seen with every other chatbot. ChatGPT has experienced several significant glitches that have required code changes, while Gemini’s attempt at maximizing diversity in its responses has led to many inaccuracies and issues. Meta’s AI tools have also been challenged with “contentious” questions, and much of that has come from expanded access and use.
Which likely means that Grok will experience the same, but we haven’t seen the same level of issues with the tool as yet because a relatively small number of people have access.
That’ll change soon, and it’ll be interesting to see how Grok and X handle these concerns with the tool.
But at the same time, Grok needs to start making money. xAI has reportedly spent tens of millions of dollars buying up hardware for the project, in the hopes of challenging OpenAI, in particular, in the AI race.
Because Elon’s pissed that OpenAI didn’t want him to be their CEO. After becoming an early investor in the project, Elon offered to take over as chief, which was rebuffed by the OpenAI team. OpenAI then became a for-profit project, and Musk is annoyed that they both took his initial donation to the company and never paid him back, while also rejecting him.
So now, he wants xAI, and Grok, to beat OpenAI. But to do that, Grok also needs to bring in users, and revenue.
Which it’s not yet.
Will these new expansions change that?
xAI says that Grok-1.5 will be made available very soon.
