{"id":816877,"date":"2025-01-03T22:12:23","date_gmt":"2025-01-04T04:12:23","guid":{"rendered":"https:\/\/newsycanuse.com\/index.php\/2025\/01\/03\/ai-briefing-writers-cto-on-how-to-make-ai-models-think-more-creatively\/"},"modified":"2025-01-03T22:12:23","modified_gmt":"2025-01-04T04:12:23","slug":"ai-briefing-writers-cto-on-how-to-make-ai-models-think-more-creatively","status":"publish","type":"post","link":"https:\/\/newsycanuse.com\/index.php\/2025\/01\/03\/ai-briefing-writers-cto-on-how-to-make-ai-models-think-more-creatively\/","title":{"rendered":"AI Briefing: Writer\u2019s CTO on how to make AI models think more creatively"},"content":{"rendered":"
\n
\n
\n

By Marty Swant<\/a><\/span> \u00a0\u2022\u00a0 January 3, 2025<\/span> \u00a0\u2022<\/p>\n<\/div>\n

<\/p>\n<\/p><\/div>\n

\n

When training data is similar across major large language models, finding ways to make them more creative and more differentiated is increasingly important. That reality has more enterprise customers asking for ways to make AI more creative when generating content \u2014 and to help with the actual process of thinking creatively.<\/p>\n

Last month, the AI startup Writer released a new LLM called Palmyra Creative<\/a> that aims to help enterprise businesses squeeze more creativity out of generative AI. The goal isn\u2019t just to help with outputs; it\u2019s also to help companies using AI in more creative ways. Palmyra Creative follows other domain-specific LLM released from Writer such as the the healthcare-focused Palmyra Med and the finance-focused Palmyra Fin<\/a>. (Writer\u2019s customers using various models include Qualcomm, Vanguard, Salesforce, Kenvue, Uber and Dropbox.) <\/p>\n

\n

In terms of creative thinking, AI models overall already have evolved quite a bit over the past few years. Some experts have found LLMs to be more creative than humans in areas like divergent thinking. Last year, researchers at the University of Arkansas published a paper<\/a> exploring how OpenAI\u2019s GPT-4 model is able to generate multiple creative ideas, find varied solutions to problems, and explore various angles. However, current LLMs still are largely limited to their own knowledge via training data \u2014 rather than lived experiences or learned lessons like humans are able to tap into.<\/p>\n

Writer\u2019s process involves creating AI models that are self-adapting or \u201cself-evolving,\u201d<\/a> said Writer CTO Waseem Al Shikh, who co-founded the company with Writer CEO May Habib in 2020. Rather than worrying about the sheer size of a model, Shikh explained the company\u2019s focus now is on developing models with a framework built around three separate buckets: model knowledge, model reasoning and model behaviors.\u00a0<\/p>\n

\u201cIt\u2019s not just enough to have a creative model,\u201d Al Shikh told Digiday in an interview last month. \u201cIt\u2019s just like a human, right? If you all just have the same libraries with a lot of books, each will come with ideas, but the funny thing is we\u2019re not just creating all the ideas with one clear theme. So the plan in the future now is to have self-evolving functionalities to all of our models and having creativity be at the top of the list.\u201d<\/p>\n

Writer\u2019s updates also benefit from the company\u2019s partnership<\/a> with Nvidia through the use of NIMs<\/a> \u2014 short for Nvidia Inference Microservices \u2014 that help simplify and speed up how AI models are deployed and scaled across various enterprise-specific uses<\/a>. In a way, NIMs serve as somewhat of a flight controller that helps decide which AI model and when to use it depending on the company, its knowledge and the desired task.\u00a0<\/p>\n

\u201cWith workflows, you know the start and the steps,\u201d Shikh said. \u201cThis concept of NIM is very futuristic, we can get there, but you\u2019ll need all these models. This is why we\u2019re building domain-specific models. You can have three or four or five specific models and they are self-evolving for customer\u2019 behaviors.\u201d<\/p>\n

Unlocking new ways to think more creatively could give marketers and others new ways to find fresh ideas, break out of AI echo chambers and escape the uniform patterns that plague many AI outputs.\u00a0Writer sees retailers potentially using Palmyra Creative for personalized marketing campaigns or enhanced loyalty programs. The models might help healthcare providers simplify patient communications, equip financial firms to create more educational tools or give B2B tech companies ideas for product-positioning and refining technical documents.<\/p>\n

This conversation has been edited for brevity and clarity.\u00a0<\/em><\/p>\n

What makes Palmyra Creative different from other models?<\/strong><\/p>\n

Our larger model and bigger models \u2014 for example finance or medical \u2014 are more focused on what we call knowledge. We want them to be accurate for every single formula and every single medicine they use. When you go to a financial model, it\u2019s about focusing on core reasoning and math equations. The behavior will change also. General models try to balance between those [knowledge, reasoning and behavior].<\/p>\n

What was different about the model development process?<\/strong><\/p>\n

Since all the models have similar architectures and similar training data, you know it\u2019s just finding similarity with the weights and how much this weight actually looks like. What we decided to do is actually take the same training data we have today, but we were more creative with the creative weights. We trained three separate models and then we started to merge the models and shuffle them between the layers. What happens then is you have a unique relation that doesn\u2019t exist within any other model. We also found out the model has interesting behaviors \u2014 the model can actually push back and doesn\u2019t follow the traditional path of everyone else because the weight is very unique to the model itself. We call it dynamic merging between the layers.\u00a0<\/p>\n

Merging a model is not a new idea, but what is new is the technique itself and the utilization of the technique. The different thing here is we are slicing the model between them and we have a specific way to make sure the relationship between them is not broken so you don\u2019t end up having a gibberish output or a strange hallucination. It\u2019s a thin line between what ends up as hallucination and what creativity looks like.<\/p>\n

Reminds me of how creativity often happens in the blurred line between fact and fiction<\/strong>.<\/p>\n

A hundred percent. But we have to define it, especially with enterprise customers. What we end up saying is we want the model to say whatever it wants, but we need the model to be careful about one thing, which we call claims. There\u2019s a difference between \u201clet me give you a crazy idea\u201d and a claim that seemed unchecked. We did a lot of work around what we call controlled claims. We don\u2019t have the source of truth [for the model] because we cannot consider for example Wikipedia the source of truth, can we? It has a lot of random stuff. We cannot consider every single thing coming from every single government on the planet to be the source of truth. But we decided to say keep the model creative, but don\u2019t claim statements.<\/p>\n

Hallucinations often come with more of the explainability question when it\u2019s having to justify itself. Is that maybe less of an issue without needing to verify claims?<\/strong><\/p>\n

Exactly. We decided to start from the root of it and control the claim \u2026 The [Palmyra] Creative model is less about knowledge and more about behavior. We think enterprises<\/a> will love this creative model to write a case study or find new use cases or to write more creative stories about how to adopt their products and how you can explain it without what sounds like AI. But controlling the claim was the biggest part. Like you said, if you don\u2019t have a claim, you don\u2019t have to explain it.\u00a0<\/p>\n

How do you guide the model for when it should evolve or be creative and when it should be consistent?<\/strong><\/p>\n

We\u2019ve been working on it since early summer. What if we could make these models think more like a human? What if the models can reflect, revolve and remember? Basically, can we get those to start working outside the training set in real-time? All the models today are still stuck to the training data \u2013 without the training data, it\u2019s really hard to get it to do anything. This is what we call self-evolving. Self-evolving models mean you don\u2019t need to teach them. The model will update their weight in real time. The model will actually reflect. And the model itself can actually ensure the information.<\/p>\n

To give you a bad example: If I say my name is Waseem and I\u2019m the president of the United States, the model will be smart enough to know, \u2018Maybe your name is Waseem, but you\u2019re not the president of the United States.\u2019 This stuff that\u2019s really important, meaning if you use it more, the model will gain more control and more knowledge. It\u2019s more high-level and takes a lot of time to explain, but it\u2019s a standard transformer design with a new feature called Memory. For each layer inside the neural network has the memory layer next to it. So you can actually talk to it and see it change.\u00a0<\/p>\n

Because the model basically will not do the same mistake twice because we know that wrong answer. It remembers the wrong [one] and will try it differently next time we think about the question. I love to tell my team, most humans \u2014 not all of us \u2014 learn from our mistakes and we don\u2019t do the same mistakes twice.<\/p>\n

Prompts & Products \u2014 AI-related news and announcements<\/strong> this week<\/strong> <\/p>\n