Pakistan's pursuit of truth blurred, as Grok distorts reality
Updated Tuesday Oct 21 2025
If you happened to tune into political conversations on X (formerly Twitter) about Pakistan the past few months, you might have come across the chatbot Grok, X’s generative AI assistant. Since its inception in 2023, Grok has become the go-to fact-checking tool for users when they come across posts that require context or verification. Of late, however, Grok has also become the tool of choice to settle political arguments, demonstrating not only an increasing over-reliance on AI for fact-checking but perhaps a shifting understanding of truth itself.
In a country where speaking truth, particularly regarding politics and society, has always been a difficult task, AI can feel like a panacea. Those delivering the news and our information diet are walking a tightrope, stating things as they are while the sword of censorship hangs above, resulting in them speaking in insinuations and euphemisms. Our institutions often gaslight us about big political events, even when we can see the reality. Particularly in this political and social milieu, the turn to generative AI chatbots as more "objective", direct or incorruptible is both understandable and tempting.
To err is AI
Turning to Grok to provide objective answers about political questions is part of an underlying belief that technology is inherently unbiased. Humans, seen as susceptible to biases, are often placed in opposition to technology, which is perceived as incorruptible. This belief, however, is completely unfounded. Artificial intelligence, particularly generative AI, relies on existing datasets and information. This data can often be extremely biased and inaccurate. There is little information on the datasets Grok draws from, but if the results are any indication, they are far from unbiased.
At the same time, when Pakistanis were turning to Grok to fact-check journalists, or journalists using it against their peers, Grok was heavily criticised for racist remarks in answers to questions by users, including referring to itself as "MechaHitler". The company behind the chatbot apologised and stated that the speech was due to a “coding issue”. Many would argue that Grok, like other generative AI tools, merely relies on information from the internet and uses pattern recognition to arrive at an answer.
If the dataset itself is heavily racist, sexist, ableist or inaccurate, it is only natural that generative AI chatbots are more likely to reproduce such answers. However, given the presumption of accuracy and objectivity we assign to technology, and by extension, these tools, these answers are more likely to be seen as more authoritative than a remark by the average user.

However, this is not merely an “input” problem. Firstly, there is often little clarity or transparency on the design of these chatbots. How is it designed to process information? What safeguards and filters have been built in? A generative AI chatbot does not treat all the information it processes equally; it is often dependent on the design of the large language model (LLM), or the system prompts built into the tool.
If Grok is programmed to give more credence to a certain type of input, that bias is more likely to reflect in its answers. Secondly, the functionality of these tools depends largely on the way data is coded and labelled for the LLM to process. This process of coding is often outsourced to low-wage workers in the global south and is invisibilised in commonly held understandings of AI. Biases can be built into the data and the tool itself at every stage.
If Grok has been designed to source information from X, the platform where it is mostly deployed, the answers will start to reflect the biases of speech on the platform. While it is unclear if this is the case, it would come as no surprise that a tool like Grok pulls from popular discourse on social media to generate its answers, which is not commonly conducive to accurate or carefully analysed answers.
AI as a fact-checker
So why do we turn to these tools to tell us the truth or mediate political debates? Technology, seen as more authoritative, can be weaponised within political debates to shut down opinions or discredit others. Just as one would draw on experiences, authoritative sources or logic while having a conversation with someone about politics, generative AI chatbots are being treated similarly. Overall, this is an alarming trend for an audience, especially in Pakistan, that lacks digital and media literacy to view these answers with scepticism.
Pakistan is not the only place where generative AI tools are used for authentication, for instance. Many users turned to Grok during the armed attacks between Iran and Israel, as they did during the Pakistan-India conflict this year. However, research found that Grok provided inconsistent answers and, in fact, contributed to misinformation. The trust we place in chatbots takes on insidious implications when used in such tense situations, where misinformation can be used to manufacture consent for more violence.

It can also lead to entrenching pre-existing biases, as users can generate desired answers through leading questions and prompts. Earlier this year, the BBC conducted a study of AI assistants answering prompts about the BBC news items and found that 51% answers had significant issues, 19% of answers introduced factual errors into the stories, and 13% sourced incorrect quotes, some of which did not exist at all.
The impact of these chatbots being used for fact-checking is amplified against the backdrop of a series of decisions by social media platforms to divest resources from content moderation. Users are left not only to their own devices in an information landscape increasingly dominated by AI-generated content but are ironically asked to use other AI tools to verify their authenticity.
Performance of truth
Grok is a unique AI app even among the increasingly saturated market of generative AI tools, given that it is pushed out aggressively to X users and allows for a public performance of fact-checking. Phrases such as “Grok is this true?” or “Grok what is this” have become common refrains under posts, often not only for the benefit of the individual user soliciting Grok but also for everyone watching.
Summoning Grok has become a way to quickly mobilise attention to support or challenge the authenticity of a post — people are watching the fact-checking unfold. However, there is much we don’t see in this process, the "under the hood" workings of the AI tool and the decisions it makes. As this practice becomes more prevalent, the implications of placing so much trust in generative AI tools must be questioned.

Encouragingly, the trend speaks to a desire to establish facts and discern the truth. It also means that users are questioning the content they see, craving tools that can tell them if what they see is real. That is a good starting point for exploring alternatives to AI chatbots. Investing in better digital literacy and investigative reporting might provide more satisfactory, albeit long-term, answers. It is an uphill task as AI is pushed so aggressively onto users by platforms and interwoven into our lives, often without consent.
The realisation that AI is deeply flawed and riddled with biases should be at the core of how we view AI. In Pakistan, as we move towards regulating AI, we must also grapple with its larger implications, such as the environmental consequences, especially in a country like Pakistan that is so vulnerable to climate change, and its impact on the economy, not simply as a boon but as fundamentally changing patterns of labour and the devastating impact on the creative economy.
AI is changing norms around how we speak, consume information, and who we trust. Not all of this can be addressed through policies and laws; we need a shift in our cultural conversation that is informed and grounded in a nuanced understanding of how AI works.
Shmyla Khan is a researcher working on the intersection of gender, technology and human rights.
Header and thumbnail image via Canva