• Aleph Alpha introduced a groundbreaking tokenizer-free (T-Free) LLM architecture that enables superior efficiency and effectiveness for fine-tuning and customization of AI across different languages, alphabets and specialized industries.
• This innovation addresses the limitations of conventional LLMs and unlocks new possibilities for sovereign AI solutions for governments and enterprises.
• The collaboration with AMD and Schwarz Digits strengthens Aleph Alpha’s new LLM architecture with high-performance computing and a sovereign cloud solution.

Aleph Alpha, a leading AI technology solutions provider headquartered in Germany, has announced a new architecture innovation for LLMs to address one of the most critical challenges in AI.

Teaching today’s popular closed- or open-source LLMs new languages or unique industry knowledge (often crucial for enterprises and governments) tends to produce underwhelming results and fine-tuning often proves ineffective. A key reason for this is that the patterns these LLMs learn are based on the tokenized version of the text they were trained on. If new text differs considerably from the original training data, it cannot be efficiently tokenized.

“Our tokenizer-free model architecture is a key solution to tackle a major challenge in AI: fine-tuning and creating sovereign models for diverse alphabets, low-resource languages and highly specific industry needs – areas where customization efforts have fallen short so far. Paired with the groundbreaking efficiency of the latest AMD AI technology, this marks a transformative leap, unlocking strategic opportunities previously out of reach,” says Jonas Andrulis, Founder & CEO, Aleph Alpha.

Aleph Alpha’s new LLM architecture does not require a tokenizer, extending LLM capabilities beyond the original training data. This approach solves a key problem of current Language Model Architectures, which most commonly can only be trained efficiently on data that uses similar tokens to their original training data. Fine-tuning current models on out-of-distribution data requires very large data sets (often not available for less common languages) and massive amounts of compute, resulting in a model with inferencing costs and an environmental footprint at an unsustainable level.

This groundbreaking development is set to lay the foundation for a new generation of sovereign AI solutions that enable governments and enterprises to build their own sovereign GenAI solutions based on their native languages, cultural specifications, and specialized knowledge and terminology. Consequently, this will democratize access to this technology and promote economic and cultural plurality.

“I founded Aleph Alpha with the mission to empower the sovereignty of countries and companies around the world in the era of AI. For our customers, this means open-source integration, maximal transparency and trustworthiness for a new era of human-machine collaboration, future-proof transformative AI solutions and free choice of execution environment,” says Jonas Andrulis, Founder & CEO, Aleph Alpha.

By combining Aleph Alpha’s cutting-edge GenAI technology with the performance and efficiency of AMD Instinct™ MI300 Series accelerators, and optimizations for the open AMD ROCm™ software stack, the company has achieved significant advancements in GenAI performance. Through leveraging the significant memory capacity of the AMD Instinct accelerators, this turnkey solution offers both performance and efficiency advantages, particularly for low-resource languages.

For example, using Finnish, this innovative AI architecture can deliver a 70 percent reduction in training cost and carbon footprint compared to alternative options (see the figure below, which presents a comparison of single-node performance measurements between T-Free and state-of-the-art tokenizer-based models).

“This collaboration brings more than AI – it delivers resiliency and innovation to the European AI ecosystem,” says Keith Strier, SVP, Global AI Markets, AMD. “We are thrilled to collaborate with Aleph Alpha and Schwarz Digits to boost Europe’s native AI capabilities and create a new AI trifecta for governments: a hyper-transparent GenAI platform, developed and trained within Europe, delivering exceptional efficiency on our AMD AI infrastructure.”

In addition to a high-performance compute platform based on AMD Instinct GPUs and the open AMD ROCm software stack, Aleph Alpha is also leveraging the AMD SiloAI team in Helsinki, which helped demonstrate the revolutionary multilingual capabilities of its new model architecture. This initiative is further strengthened by the data center and technical delivery capabilities of STACKIT, the sovereign cloud of Schwarz Digits, the IT and Digital Division of Schwarz Group, Europe’s largest retailer with 575,000 employees, based in Germany.

Aleph Alpha’s Pharia Operating System’s AI capabilities, now optimized on AMD, create a full-stack sovereign offering that combines AI software and hardware. This will deliver significant benefits in terms of computing costs and environmental footprint. In addition to the new Pharia LLM generation of tokenizer-free foundation models pioneered by Aleph Alpha, PhariaAI incorporates several unique innovations that enable unprecedented levels of explainability, transparency, and efficiency, facilitating the rapid integration of trustworthy GenAI into industry and government operations.

Our latest peer-reviewed research paper on tokenizer-free (T-Free) language modelling is now available here.

For more technical details, see our Blog here.

For more information about AMD Instinct Accelerators” line at the end of the blog: AMD Instinct™ Accelerators.

AMD, the AMD logo, AMD Instinct, ROCm, and combinations thereof are trademarks of Advanced Micro Devices, Inc.

Aleph Alpha was founded in 2019 with the mission to research and build the foundational technology for an era of strong AI. The team of international scientists, engineers, and innovators researches, develops, and deploys transformative AI like large language and multimodal models and runs the fastest European commercial AI cluster. Its generative AI solutions are the prime choice for enterprises and governmental institutions seeking to retain independence, secure their data, and build trustworthy solutions.

Find out more: https://aleph-alpha.com/de/

Press Contact: press@aleph-alpha.com

Schwarz Digits is the IT and digital division of Schwarz Group. It offers compelling digital products and services that meet Germany’s high data protection standards. Schwarz Digits thus guarantees the greatest possible digital sovereignty. With this claim, Schwarz Digits provides the IT infrastructure and solutions for the extensive ecosystem of the companies of Schwarz Group and develops it further for the future. Schwarz Digits creates optimal conditions for the development of trend-setting innovations for end customers, companies and public sector organizations. Schwarz Digits includes 7,500 employees of the brands Schwarz IT, Schwarz Digital, STACKIT, XM Cyber, Lidl e-commerce, Kaufland e-commerce, Schwarz Media and mmmake.

The cloud and colocation provider STACKIT is part of Schwarz Group. External partners and customers in the DACH region can also rely on the cloud services that Schwarz Group companies have been benefiting from for years when it comes to digital transformation. With data sovereignty that goes far beyond the market standard and individual approaches to the implementation and operation of cloud solutions, STACKIT provides holistic support for digitalization projects. Headquartered in Neckarsulm the team is paving the way for an independent Europe – digital, leading. STACKIT belongs to the IT and digital division of Schwarz Group, Schwarz Digits.