EtusivuYleiset julkaisuWhat AI Still Cannot Do

AI has become increasingly popular in many industries. But the real question is: how trustworthy are the answers?

AI responses tend to be most reliable for well-established, widely documented knowledge: explaining concepts, standard methods, common code patterns, and summarizing text you provide. They are least reliable for specific facts (precise figures, citations, dates), recent events, and anything that requires judgment on a real-world situation the model cannot verify.

Below are six areas where current AI systems still fall short, and how to work around them.

Accuracy

When a large language model produces confident text that is wrong, this is called hallucination: plausible but nonsensical or unfaithful generated content (Ji et al., 2023). Hallucinations can be divided into two forms. Factuality hallucinations occur when the output conflicts with real-world facts or generate unverifiable details. Faithfulness hallucinations are when the output doesn’t match the source material such as the instructions, or its own earlier statements (Huang, L. et al., 2025).

The frequency of the hallucinations heavily relies on the task. To illustrate, hallucination rates for leading models are roughly 1 to 2 percent when the model summarizes a document (Vectara HHEM leaderboard, 2025). Open questions answered from memory are far less reliable. In the legal domain, general-purpose models hallucinated on 58 percent (GPT-4) to 88 percent (Llama 2) of specific, verifiable questions about federal court cases (Dahl et al., 2024). Even commercial legal research tools built on retrieval and marketed as hallucination-free still produced hallucinations in 17 to 33 percent of queries (Magesh et al., 2025).

How to mitigate that?

  • Attach your own material to the prompt. Paste the source document instead of asking from memory.
  • Only trust AI-supplied references after checking them against the original source.
  • Ask the model to say “I don’t know” when uncertain, and treat a confident tone as no signal of correctness.
  • Verify every number and fact that will carry weight in a decision or publication.

Reasoning

AI usually produces output by matching the patterns from training data rather than by systematic reasoning. When researchers tested multi-digit multiplication, models performed well only on problem sizes resembling their training data and did not generalise to larger ones even after fine-tuning. Performance also decayed quickly with complexity, since an error in an early step propagates into all the steps that follow (Dziri et al., 2023).

The reasoning can also be fragile in ways a human’s is not. Adding a single irrelevant sentence to a math problem dropped performance by up to 65 percent on the smallest models tested, and by roughly a third even on the strongest. Changing only the numbers in a problem caused smaller but consistent declines (Mirzadeh et al., 2024).

Models also cannot reliably check their own work. Without external feedback, asking a model to review and correct its own reasoning often fails to improve the answer and can even make it worse (Huang, J. et al., 2024). And newer “reasoning” models are not automatically more accurate: OpenAI’s o3 hallucinated twice as often as its predecessor on one factual benchmark, because it makes more claims overall (OpenAI, 2025).

How to mitigate that?

  • Break complex problems into small steps and verify each step yourself.
  • Check AI calculations against hand calculations or known special cases.
  • Use external verification wherever it exists: run the code, test the formula, compare against published results.
  • Do not ask the model to grade its own output. Use a separate review pass by a human or a different model.

Robustness

AI performance is fragile in ways that have little to do with the difficulty of the task. Rewording a question or adding an irrelevant detail can change the answer, as the drops in mathematical accuracy described above show (Mirzadeh et al., 2024). Position matters too: accuracy is highest for information at the beginning and end of a long document and degrades significantly for content in the middle, a phenomenon known as “lost in the middle” (Liu et al., 2024). So does language. Across 29 languages on a parallel benchmark, accuracy in low-resource languages fell by up to 24 percentage points relative to high-resource ones (Xuan et al., 2025)

How to mitigate that?

  • Place the most important information at the start or end of long prompts.
  • Test the same question with different phrasings and compare answers.
  • Keep prompts and inputs as short and focused as the task allows.
  • Be extra careful when working in languages other than English.

Knowledge

A model’s knowledge has two kinds of gap. The first is temporal: knowledge is frozen at the training cutoff, so the model does not know about recent events, new product versions, updated regulations, or current prices unless it is connected to search or given the information directly. The second is a matter of density. A model’s ability to answer a factual question depends directly on how many relevant documents appeared in its training data, so rarely documented topics are learned poorly no matter when they were written about (Kandpal et al., 2023). Moreover, training rewards confident guessing over answering with uncertainty so the model will often fill those gaps with convincing fabrication rather than saying it does not know (Kalai et al., 2025).

How to mitigate that?

  • For anything time-sensitive, use a model with web access or supply current sources yourself.
  • Check the publication date of anything the model tells you about standards, laws, or software.
  • For niche domains, treat the model as a starting point, not an authority. Confirm with primary sources.

Security

AI-generated code frequently contains security flaws. In controlled testing, roughly 40 percent of the programs GitHub Copilot generated were vulnerable (Pearce et al., 2023). A separate analysis of AI-generated code already committed to real GitHub projects found weaknesses in about a quarter to a third of snippets, spanning 43 vulnerability categories (Fu et al., 2025). The risk is not limited to the code itself. Models are also vulnerable to prompt injection, where instructions hidden in a document or web page override what the user asked for, and researchers have demonstrated attacks that use this to make a browsing assistant send user data to an attacker (Greshake et al., 2023).

How to mitigate that?

  • Review and security-test all AI-generated code before deployment. Treat it like code from an unknown contributor.
  • Run static analysis and dependency checks as a standard step.
  • Never paste secrets, credentials, or confidential data into prompts unless your organization’s policy explicitly allows it.
  • Be cautious when models process untrusted external content.

Ethics

Social biases found in training data are learned, reinforced, and amplified by models, which may appear as stereotypes in recruiting, assessment, and creative activities (Gallegos et al., 2024). They also cannot take responsibility for their According to COPE’s position statement, AI technologies are unable to fulfil authorship standards since they are unable to assume accountability for the work (COPE, 2023). Furthermore, the ICMJE recommendations state that humans are ultimately responsible for reviewing and ensuring the accuracy of any content produced with AI assistance, including originality and citation integrity (ICMJE, 2025). There are also open questions around copyright of training data, environmental cost, and the effect of AI-generated content on trust in information.

How to mitigate that?

  • Keep a human accountable for every AI-assisted output. The tool assists; the person answers for the result.
  • Disclose AI use where norms or policies require it, especially in academic and professional publishing.
  • Review outputs for bias, particularly in decisions affecting people.
  • Keep conceptual decisions, judgment calls, and interpretation with the human. Use AI for drafting, coding support, and ideation, with verification built into the workflow.

The bottom line

None of these limitations means AI is useless. It means AI is a powerful but unreliable collaborator. The evidence points to one consistent conclusion: the systems work best when a knowledgeable human stays in control of the decisions, verifies the outputs, and treats the AI as a tool rather than an authority.

If you would like to explore the topic in more detail, you can read our publication AI-Assisted Simulation for Mass Customization: A Conceptual Framework for SMEs at the link below:

https://urn.fi/URN:NBN:fi-fe20260818118151

Author

Sorayya Amirahmadi, Project Specialist, Savonia University of Applied Sciences

This article was prepared with the help of AI.

References

COPE (2023). Authorship and AI tools: COPE position statement. Committee on Publication Ethics. https://publicationethics.org/cope-position-statements/ai-author

Dahl, M., Magesh, V., Suzgun, M., & Ho, D. E. (2024). Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models. Journal of Legal Analysis, 16(1), 64–93.

Dziri, N., et al. (2023). Faith and Fate: Limits of Transformers on Compositionality. Advances in Neural Information Processing Systems 36 (NeurIPS 2023).

Fritz, M. (2024) Fundamental Risks in the Current Deployment of General-Purpose AI Models: What Have We (Not) Learnt From Cybersecurity? arXiv. doi:10.48550/arXiv.2501.01435.

Fu, Y., Liang, P., et al. (2025). Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study. ACM Transactions on Software Engineering and Methodology. https://doi.org/10.1145/3716848

Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024). Bias and Fairness in Large Language Models: A Survey. Computational Linguistics, 50(3), 1097–1179. https://doi.org/10.1162/coli_a_00524

Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., & Zhou, D. (2024). Large Language Models Cannot Self-Correct Reasoning Yet. International Conference on Learning Representations (ICLR 2024).

Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., & Liu, T. (2025). A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Transactions on Information Systems, 43(2), 1–55. https://doi.org/10.1145/3703155

ICMJE (2025). Recommendations for the Conduct, Reporting, Editing, and Publication of Scholarly Work in Medical Journals: Artificial Intelligence. International Committee of Medical Journal Editors. https://www.icmje.org/recommendations/browse/artificial-intelligence/

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys, 55(12), 1–38. https://doi.org/10.1145/3571730

Kalai, A., Nachum, O., Vempala, S., & Zhang, E. (2025). Why Language Models Hallucinate. arXiv:2509.04664.

Kandpal, N., Deng, H., Roberts, A., Wallace, E., & Raffel, C. (2023). Large Language Models Struggle to Learn Long-Tail Knowledge. Proceedings of the 40th International Conference on Machine Learning (ICML), PMLR 202, 15696–15707.

Liu, N. F., et al. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173.

Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D., & Ho, D. E. (2025). Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. Journal of Empirical Legal Studies, 22, 216–242. https://doi.org/10.1111/jels.12413

Mirzadeh, I., et al. (2024). GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models. arXiv:2410.05229. Published at ICLR 2025.

OpenAI (2025). OpenAI o3 and o4-mini System Card. 16 April 2025. https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f3722c1/o3-and-o4-mini-system-card.pdf

Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., & Karri, R. (2023). Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions. Communications of the ACM, 68(2). https://doi.org/10.1145/3610721

Vectara (2025). Hughes Hallucination Evaluation Model (HHEM) Leaderboard. https://github.com/vectara/hallucination-leaderboard

Veracode (2025). 2025 GenAI Code Security Report. https://www.veracode.com/resources/analyst-reports/2025-genai-code-security-report/

Veracode (2026). Spring 2026 GenAI Code Security Update. https://www.veracode.com/blog/spring-2026-genai-code-security/

Xuan, W., et al. (2025). MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation. arXiv:2503.10497. Published at EMNLP 2025.

EU:n lippu, jossa lukee ”Euroopan unionin osarahoittama”, sekä musta kilpimerkki, jossa on keltainen kuvio ja kruunu, tekstin ”Pohjois-Savon liitto” vieressä vaalealla taustalla.