Meet, ORCA: Microsoft and OpenAI’s New AI Love-Child…

Microsoft, in partnership with OpenAI, has unveiled a revolutionary artificial intelligence (AI) model named Orca. It’s like ChatGPT 4, and it could one day by smarter too…

Microsoft and OpenAI’s bromance continues with the creation of a brand new AI model called ORCA. This newly announced AI platform is engineered to emulate and learn from larger foundational models such as GPT-4, thereby circumventing the constraints typically associated with smaller AI models.

Despite its compact size, Orca boasts the ability to be tailored for specific tasks and function autonomously, thereby negating the necessity for substantial computational resources and expansive data centers.

What is Microsoft’s ORCA AI?

Orca is a sophisticated AI model, equipped with 13 billion parameters, that can replicate the intricate reasoning and complex instructional processes of GPT-4, a behemoth believed to house over one trillion parameters.

Furthermore, Orca is constructed on the foundation of the Vicuna model and has demonstrated exceptional learning capabilities. Microsoft is harnessing the power of large-scale and diverse imitation data to stimulate progressive learning in Orca, a strategy that could reshape the landscape of AI learning.

Recent research has focused on enhancing the capability of smaller models through imitation learning, drawing on the outputs generated by large foundation models (LFMs).

A number of issues impact the quality of these models, ranging from limited imitation signals from shallow LFM outputs; small scale homogeneous training data; and most notably a lack of rigorous evaluation resulting in overestimating the small model’s capability as they tend to learn to imitate the style, but not the reasoning process of LFMs.

To address these challenges, we develop Orca, a 13-billion parameter model that learns to imitate the reasoning process of LFMs. Orca learns from rich signals from GPT 4 including explanation traces; step-by-step thought processes; and other complex instructions, guided by teacher assistance from ChatGPT.

Microsoft Research

ORCA Performance & Capabilities

When it comes to performance, Orca has already made a significant impact. It has outstripped Vicuna by a staggering 100% on demanding zero-shot reasoning benchmarks like Big-Bench Hard (BBH).

Additionally, it operates at a speed that is 42% faster than traditional AI models on AGIEval, a testament to its efficiency and computational prowess. In a head-to-head comparison with ChatGPT on reasoning benchmarks, Orca exhibits competitive performance, a remarkable feat considering its smaller size.

While it has shown potential on academic exams such as the SAT, LSAT, GRE, and GMAT, it is important to note that it still trails behind GPT-4.

Is ORCA As Good As ChatGPT 4?

Pin

The research team at Microsoft anticipates that Orca’s abilities and capabilities will continue to evolve and improve. This will be facilitated by learning from detailed explanations provided by humans and more advanced language models.

In simpler terms, imagine Orca as a highly intelligent student who, despite being younger and smaller, can learn from and mimic the reasoning abilities of older, more experienced students (like GPT-4).

This ability allows Orca to perform tasks and solve problems that typically require larger, more resource-intensive models.

The implications of this are vast, as it means that advanced AI capabilities can be made more accessible and efficient, potentially revolutionizing sectors like education, healthcare, and technology.

This development is a testament to the power of AI and its potential to transform our world. By making AI models more efficient and adaptable, we can harness their capabilities in more diverse and innovative ways, opening up new possibilities for their application.