How to Measure AI Literacy: Knowledge Checks and Certificates

You measure AI literacy with a short knowledge check (a 5 - 10 question test) in which the employee must achieve a clearly defined passing score, and you confirm the successful result with an AI literacy certificate that includes the date and name. A knowledge check can add useful information beyond attendance, helping identify what the employee understood and which topics need further attention. Article 4 does not prescribe a test or guaranteed individual literacy level. It is the measurable result (test score and certificate) that becomes the document you can present as evidence under Article 4 of the EU AI Act (Regulation (EU) 2024/1689).

The measurement should be simple but verifiable: questions linked to real work situations, results saved, and the certificate clearly showing who passed the check and when.

What Does “Measuring” AI Literacy Mean?

AI literacy assessment is a way to assess an employee’s understanding and plan further support for the AI literacy measures covered by the EU AI Act (Regulation (EU) 2024/1689). Article 4, applicable since 2 February 2025, obliges providers and deployers to take measures to support the development of AI literacy among staff and relevant persons, taking into account their knowledge, experience, training, context, and the people who may be affected. The article does not prescribe a specific methodology or a guaranteed level of literacy, so you choose the measurement method yourself - the key is that it is justified, documented, and proportionate to your operational context.

In practice, measurement is based on three components:

• Knowledge check - a test or task that demonstrates whether the employee understood the material.
• Passing score - a pre-defined score for completing this particular check, not a legal certification of literacy.
• Certificate and record - a document recording who, when, and with what result passed the check.

If you are not yet sure what the concept itself entails, start with the article AI Literacy - What It Is and Why It Is Mandatory, then return to measurement.

Why Participation Alone Is Not Enough

Many small businesses mistakenly believe that organising training alone proves compliance. However, an email with an attached PDF or “we discussed this in a meeting” is not a measurable result. If you need to demonstrate compliance, you must be able to answer the question: how do you know the employee understood the material?

The knowledge check closes this gap. It turns the intangible “literacy” into a concrete number - a score that can be compared against a threshold. Additionally, the test reveals weaknesses: if most of the team gets a question about entering personal data into public tools wrong, you know exactly which topic needs reinforcement. More on common pitfalls and how to avoid them can be found in the article Common AI Literacy Mistakes.

What Questions to Include in the AI Knowledge Check?

A good knowledge check covers the same topic groups as the training. The table below shows a minimal set of questions for a small company, with examples of what each question should verify.

We recommend 5 - 10 questions - enough to assess understanding without overburdening the employee. Use closed-ended questions (multiple choice) as they are easy to evaluate objectively and automate. Avoid questions that can be answered by guessing or that only test memorisation - a good question is only solved correctly if the employee truly understood the material.

Practical tip: tailor the questions to your business. For an accounting firm, it is relevant to check whether employees know that client financial data must not be entered into public tools; for a marketing team, whether they understand the risks of copyright and bias when generating texts and images. A generic test “from the internet” checks less than it seems because it is not linked to the tasks your team performs daily. Good measurement always relies on real AI usage scenarios in your company - the same ones you collected when inventorying tools.

What Should the AI Literacy Passing Score Be?

There is no legally defined passing score - you set it based on risk level and your company’s needs. A practical rule for small businesses:

1. Example internal threshold - 80%. Out of 10 questions, the employee must answer at least 8 correctly. This is high enough to rule out random guessing but realistic.
2. Option to retake. If the threshold is not met, the employee returns to the material and retakes the test. The goal is not to “fail” but to raise literacy.
3. Mandatory critical questions. For certain questions (e.g., about entering personal data), you can require that they be answered correctly regardless of the overall score.

The most important thing is to set the threshold in advance and apply it uniformly to everyone. This makes the assessment objective and allows you to explain why a particular employee passed this specific check. If the threshold were chosen “by feel” for each person individually, you lose the point of measurement - the result could neither be compared nor defended.

Also, ensure the check does not become a formality that everyone “somehow” passes. If every employee scores 100% on the first try, the questions are likely too easy and measure nothing. A well-calibrated check occasionally shows that someone did not understand the material - and that is exactly when it does its job: it redirects the person back to training until literacy becomes real, not just recorded on paper.

What Should an AI Literacy Certificate Look Like?

An AI literacy certificate is one practical record of a completed learning activity, linking a specific person to a passed check. A proper certificate should include the following elements:

• Employee’s first and last name - to clarify who completed the training;
• Training title and brief content - what exactly the programme covered;
• Knowledge check result or at least confirmation of passing - proof that it was not just attendance;
• Date - when it was completed (important for tracking annual updates);
• Issuing company - who is responsible for the training.

The certificate does not need to be a beautifully designed document - the content and its link to the record are what matter. How certificates fit into the overall training organisation is described in more detail in the article AI Literacy Training for Employees, and a detailed programme plan with modules and duration is in the article AI Training Programme for Companies.

How to Record Results in a Compliance Log?

Individual certificates are useful, but the real value from an audit perspective is created by a consolidated record - a single place where the entire team’s status is visible. A minimal record for a small company looks like this:

Such a record immediately answers the questions any inspector might ask: are all users trained, was the check passed, when is a retake needed? Remember that the AI field changes rapidly - general-purpose AI model rules began applying on 2 August 2025, subject to transitional provisions, and most of the remaining obligations under the regulation apply from 2 August 2026, so annual updates and their recording are part of the measurement, not a one-off action.

Can Measurement Be Automated?

Yes - and for small businesses, this is the most practical approach. Manual test grading, certificate filling, and record management take time and are prone to errors. An automated process presents the questions, calculates the score against the set threshold, immediately issues a certificate, and records the result in the consolidated log.

This exact closed loop is offered by nebedirbam.lt: together with the compliance document package, for a fixed price of €149 + VAT (€299), you get AI literacy training with a knowledge check and certificates, plus an AI usage policy, tool registry, compliance log, and employee AI guide. These documents are practical aids for demonstrating compliance but are not legally mandated and do not guarantee legal compliance. More information is available on the homepage.

If you first want to check your own knowledge without the full document package, there are also free AI literacy training with a knowledge check - a good opportunity to assess the current level before moving to full compliance. The free path is for one signed-in participant; team training is included in the paid documentation package.

Quick Summary

AI literacy assessment in a small business is based on three things: a knowledge check (5 - 10 question test), a pre-defined passing score (usually 80%), and an AI literacy certificate that records the name, date, and result. A measurable result and a consolidated record show who took this check, when and with what score. They can support review of the measures taken, but do not themselves establish full Article 4 compliance. A broader collection of related topics can be found in the list of all articles.

The requirements of Article 4 of the EU AI Act are outlined on the official European Commission page AI Act Service Desk - Article 4, and the general timeline for the regulation’s application is available on Regulatory Framework on AI. These sources provide up-to-date information on AI literacy development measures and the regulation’s effective dates.