The Ministry of Science and ICT, together with the National Information Society Agency, has announced that they will publicly recruit institutions to carry out the “Performance Evaluation Dataset Construction Project” from July 17 to August 7.
The purpose of this project is to construct datasets that align with domestic conditions to objectively evaluate the performance of “Indigenous AI Foundation Models.” The total budget for the project is 2.4 billion won, and three tasks will be selected, each receiving 800 million won in support.
Although generative AI services are spreading worldwide, the criteria used for performance evaluation are mainly focused on English-speaking regions. This has led to criticism that domestic usage environments and social characteristics are not properly reflected.
In response, the Ministry of Science and ICT has established dataset evaluation criteria by reflecting the opinions of domestic and international experts. The construction targets for this year include a dataset for evaluating mathematical problem-solving ability (mathematics field), a question-and-answer and reasoning dataset that assesses Korean knowledge (knowledge field), and a dataset for evaluating various task performances based on long text context (long comprehension field).
In the future, the plan is to expand into datasets for evaluating various AI models, such as multimodel and agent-based AI. Companies or institutions wishing to participate in the project must possess large-scale data-based AI development capabilities, such as in hyper-scaled AI, natural language processing, and multimodal AI. They must also participate in a consortium form with at least one related institution.
Kim Kyung-man, Director of the Artificial Intelligence Policy Division at the Ministry of Science and ICT, stated, “Building a dataset that reflects the domestic environment is essential for accurately diagnosing the performance of indigenous AI models. The dataset developed from this project will be made available to elite teams and domestic AI development institutions to enhance the overall competitiveness of the ecosystem.”
Applications will be processed online through the Integrated Management System for Digital Proposals, and inquiries can be directed to the AI Data Diffusion Team at the National Information Society Agency.