YTL AI Labs Releases the First MalayMMLU
YTL AI Labs Releases the First MalayMMLU
Professor Ir. Dr Chan Chee Seng, Dean of the Faculty of Computer Science and Information Technology at Universiti Malaya, who co-led the development of the MalayMMLU with Foong Chee Mun, CEO of YTL AI Labs added, "Acceptance of the MalayMMLU at the EMNLP 2024 conference marks a significant milestone in the development of LLMs tailored to the Malaysian context. We now have a benchmark for the Bahasa Melayu language which hitherto did not exist. We believe that we will now begin to see a proliferation of applications built on the back of this benchmark. It is a great privilege for Universiti Malaya to be working with YTL AI Labs to be the first in the world to set a universal benchmark for our national language."YTL AI Labs, in collaboration with Universiti Malaya (UM), has developed the first MalayMMLU (Massive Multi-task Language Understanding) which has recently been accepted at the prestigious Empirical Methods in Natural Language Processing 2024 (EMNLP 2024) conference. The MalayMMLU is a comprehensive benchmark designed to evaluate large language models (LLMs) in Bahasa Melayu. By providing a standardised mechanism for evaluating LLMs across multiple language tasks, the MMLU benchmark facilitates fair and comprehensive assessments, driving excellence in LLM development. The MalayMMLU consists of over 24,000 questions covering the entire K-12 spectrum across 22 subjects from the Malaysian education curriculum.
YTL AI實驗室與馬來亞大學(UM)合作，開發了首個MalayMMLU（Massive Multi-task Language Understanding），該項目最近被著名的自然語言處理方法會議EMNLP 2024接受。MalayMMLU是一個綜合性基準，旨在評估馬來語大型語言模型(LLMs)。通過爲評估LLMs提供標準化機制，MMLU基準促進公平和全面的評估，推動LLM發展的卓越。MalayMMLU包含超過24,000個問題，涵蓋了馬來西亞教育課程的整個K-12領域中的22個科目。
This benchmark is crucial for improving LLMs' understanding of Bahasa Melayu and advancing applications in education, healthcare, and public services, fostering culturally relevant AI solutions for Malaysia and Southeast Asia. YTL AI Labs and Universiti Malaya will be publicly releasing the MalayMMLU dataset along with the evaluation code, enabling the broader AI community to contribute to the development of more inclusive and effective AI technologies.
這一基準對於改善LLMs對馬來語的理解，推進教育、醫療保健和公共服務領域的應用至關重要，爲馬來西亞和東南亞提供具有文化相關性的人工智能解決方案。YTL AI實驗室和馬來亞大學將公開發布MalayMMLU數據集以及評估代碼，讓更廣泛的人工智能社區能夠爲更具包容性和有效性的人工智能技術的發展做出貢獻。
Dato Seri Yeoh Seok Hong, Managing Director of YTL Power International said, "Malaysia is rapidly building an ecosystem that will support the development and adoption of AI in the country. Our first AI data centres are being built and our universities are focusing on building AI talent. The release of the MalayMMLU is an exciting milestone that will undoubtedly accelerate Malaysia's journey to becoming an AI Nation. YTL AI Labs is proud to be part of this journey."
YTL Power International董事長楊錫漢表示：「馬來西亞正在迅速建立一個支持該國人工智能發展和採用的生態系統。我們正在建設第一批人工智能數據中心，我們的高校正致力於培養人工智能人才。MalayMMLU的發佈是一個激動人心的里程碑，無疑將加快馬來西亞成爲人工智能之國的旅程。YTL AI實驗室很榮幸能夠成爲這一進程的一部分。」
Professor Ir. Dr Chan Chee Seng, Dean of the Faculty of Computer Science and Information Technology at Universiti Malaya, who co-led the development of the MalayMMLU with Foong Chee Mun, CEO of YTL AI Labs added, "Acceptance of the MalayMMLU at the EMNLP 2024 conference marks a significant milestone in the development of LLMs tailored to the Malaysian context. We now have a benchmark for the Bahasa Melayu language which hitherto did not exist. We believe that we will now begin to see a proliferation of applications built on the back of this benchmark. It is a great privilege for Universiti Malaya to be working with YTL AI Labs to be the first in the world to set a universal benchmark for our national language."
馬來亞大學計算機科學與信息技術學院院長Chan Chee Seng教授及YTL AI實驗室CEO馮志文共同領導開發MalayMMLU。他們表示：「MalayMMLU在EMNLP 2024會議上的接受標誌着爲馬來西亞背景量身定製的LLMs發展邁出的重要一步。現在，我們有了一個迄今不存在的馬來語基準。我們相信，我們現在將開始看到基於這一基準構建的應用程序的興起。馬來亞大學有幸與YTL AI實驗室合作，成爲世界首個爲我們的國家語言設立通用基準的機構。」
譯文內容由第三人軟體翻譯。