What You Can Learn From Shopify’s CEO’s Memo On Workforce AI

What Shopify’s CEO Gets Right — And Wrong — About Workforce AI Shopify’s CEO Tobias Lütke released an internal email he’d sent to employees with the subject, “AI usage is now a baseline expectation.” His enthusiastic missive implores employees to adopt AI tools “as a thought partner, deep researcher, critic, tutor, or pair programmer.” His ultimate conclusion? “AI will totally change Shopify, our work, and the rest of our lives.” Lütke says the company needs to embrace AI to keep up with its explosive growth: “In a company growing 20–40% year over year, you must improve by at least that every year.” Shopify doesn’t want to hire more employees, however: While the company has grown at least 21% per year since 2022, the number of employees has declined from 11,600 in 2022 to 8,300 in 2023 to 8,100 at the end of 2024. So this is an organization dedicated to efficiency. Reading Lütke’s memo reveals some important lessons about workforce AI, including several that you should emulate and others that you should avoid. What Shopify Gets Right: Vision, Practice, And Community You should take inspiration from several Shopify beliefs and practices for workforce AI: Develop a vision. Lütke writes: “You’ve heard me talk about AI in weekly videos, podcasts, town halls, and [Summit, a Shopify event].” Executive leadership is crucial to workforce AI efforts: Demystifying myths (such as “AI will steal my job if I use it”), establishing the benefits to both the organization and to employees, and painting a picture of the future state are all crucial to driving adoption success. For example, leaders can position AI as an opportunity builder for employees — taking boring, predictable tasks off their plates, improving their competitiveness in the job market, and solving hard problems. Empower learning on the job. Lütke writes: “Using AI well is a skill that needs to be carefully learned by using it a lot.” On-the-job experience is central to successfully learning how to use generative AI. Employees must apply AI to tasks along their daily employee journey that can benefit from the productivity-enhancing impact of genAI. They will learn from both their successes and failures. Encourage social learning. Lütke writes: “Share what you learned […] Slack and Vault have lots of places where people share prompts that they developed.” Social learning is twice as important as formal (think classroom) learning. Clients report that vigorous peer-to-peer and champions programs are central to successful genAI adoption. Many employ active Slack or Teams channels dedicated to workforce AI, too. What Shopify Gets Wrong: Expectations And Learning Styles There are a few beliefs and practices in Shopify’s memo that you should avoid, however: Be judicious, not reflexive, about using AI. Lütke writes: “Reflexive AI usage is now a baseline expectation at Shopify.” Our artificial intelligence quotient (AIQ) research shows that not all employees have the understanding, skills, and ethical awareness needed to use genAI appropriately. High-AIQ employees know when to use it and when not to use genAI for their work tasks. You can’t prove a negative. Lütke writes: “Before asking for more headcount and resources, teams must demonstrate why they cannot get what they want done using AI.” Demonstrating a financial business case for genAI is actually quite challenging today. Most AI augments rather than replaces human labor at this stage of the market, and “proving” that something doesn’t exist is hard to do. Learning requires a system of engagement. Lütke writes: “Learning is self-directed.” Learning workforce AI requires you to build a comprehensive learning system that combines formal learning, social learning, and on-the-job experience. Even sophisticated companies that rely on purely self-directed approaches are less successful than those that create iterated, reinforcing learning opportunities. Set realistic productivity expectations. Lütke writes: “Get 100x the work done” and “What would this area look like if autonomous AI agents were already part of the team?” While agentic AI is on the rise, its promise remains mostly in the future. Most of today’s AI assistants save more modest amounts of labor time. Setting appropriate expectations and KPIs, and measuring them continuously, sets you up for success. Inflated hyberbole doesn’t. Next Steps For You? Let’s Talk. I’ve spoken with over 200 organizations about workforce AI. Forrester clients should reach out to schedule a guidance session with me, and I’ll help you design a successful workforce AI strategy. source

What You Can Learn From Shopify’s CEO’s Memo On Workforce AI Read More »

7 risk management rules every CIO should follow

“In fact, CIOs often confuse risk management with compliance or cybersecurity, yet risk is much broader,” she says, advising IT leaders designate an enterprise risk officer who can serve as the CIO’s best ally, helping to navigate risks, accelerate strategic initiatives, and provide guidance on where caution is needed versus where speed is possible. Risk management is among the most misunderstood yet valuable aspects of leadership, Saibene observes. When CIOs embrace risk frameworks, they can proactively identify IT-related risks, propose mitigation strategies, and collaborate effectively with risk officers. “This not only strengthens executive buy-in, but also accelerates progress,” she explains. Rule 2: Inventory applications The most critical risk management rule for any CIO is maintaining a comprehensive, continuously updated inventory of the organization’s entire application portfolio, proactively identifying and mitigating security risks before they can materialize, advises Howard Grimes, CEO of the Cybersecurity Manufacturing Innovation Institute, a network of US research institutes focusing on developing manufacturing technologies through public-private partnerships. source

7 risk management rules every CIO should follow Read More »

$40B into the furnace: As OpenAI adds a million users an hour, the race for enterprise AI dominance hits a new gear

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More In a move that surprised the tech industry Monday, OpenAI said it has secured a monumental $40 billion funding round led by SoftBank, catapulting its valuation to an unprecedented $300 billion — making it the largest private equity investment on record. The landmark investment underscores the escalating significance of AI, and also signals a shift in the enterprise technology landscape. With such a vast war chest, OpenAI now has much more staying power in its battle to serve companies with sophisticated generative AI solutions — where it is going against giant competitors like Google and AWS, as well navigating a sensitive relationship with its partner Microsoft. It is also facing tough competitors like Anthropic and Elon Musk’s xAI. Before this round closed, questions remained around whether OpenAI had the capital to continue to play in the big leagues. Spending by companies on generative AI is expected to hit $644 billion this year, according to research company Gartner. That’s 76 percent more than was spent last year, and shows why the race is on among large companies to grab market share. In its announcement, Open AI said it now has 500 million active weekly users, a significant jump from the 400 million number it cited just a month ago. With such viral growth, the company badly needed capital to build the servers and other infrastructure to keep up with this demand. It also shows that the intense competition, where other providers such as Google, Anthropic and even Chinese companies like DeepSeek are offering AI models that often match the functionality of OpenAI’s own leading models, has not slowed OpenAI’s growth rate. In another significant twist Monday, OpenAI also announced that it planned to launch an open-weights reasoning model, and that it would allow developers to run it on their own hardware, the departure from OpenAI’s cloud subscription model that has so far driven its revenue. The funding details: a closer look For decision-makers navigating this rapidly evolving environment, understanding the implications of OpenAI’s latest financial maneuver is important. The $40 billion infusion came primarily from SoftBank, with contributions from Microsoft, Coatue, Altimeter, and Thrive Capital, according to reporting by CNBC. The capital is earmarked for OpenAI’s AI research, computational infrastructure, and enhancing its suite of AI tools, including the widely adopted ChatGPT, according to OpenAI’s post on the news. Notably, $18 billion of this funding is allocated to the Stargate project — a joint venture between OpenAI, SoftBank, and Oracle — aimed at developing extensive AI infrastructure. The reports also suggested that the latest OpenAI funding would come in several tranches, and that part of it depends on OpenAI turning into a for-profit company by the end of this year. While OpenAI is still driving a significant loss, the company projects that it will make enough revenue to break even by 2029 and then start making significant profits. CEO Sam Altman tweeted Monday morning that the company had added “one million users in the last hour,” contrasting it with the million added in the five days after ChatGPT launched 26 months ago. The latest viral surge of usage comes from the big update OpenAI made last week to its image generating technology, which has taken image creation to a whole new level of ease and sophistication — with consumers going crazy making selfies in the style of Studio Ghibli. Notably, OpenAI announced Monday that it was restoring its offer to allow free users to access the new image generating technology, something it had taken back temporarily last week after usage overwhelmed the company’s servers. While a lot of excitement around OpenAI remains on the consumer side, for enterprise technology leaders the funding development also carries big implications: OpenAI’s bolstered resources will help it fast-track the development of advanced AI models and products for enterprise as well, allowing it to stay ahead amid increased competition. Enterprises should anticipate a continued flurry of new AI-driven solutions, necessitating continued vigilance among enterprise companies to stay on top of these releases in order to remain competitive. source

$40B into the furnace: As OpenAI adds a million users an hour, the race for enterprise AI dominance hits a new gear Read More »

Trump's FCC Nominee Faces Light Scrutiny At Senate Hearing

By Nadia Dreid ( April 9, 2025, 9:50 PM EDT) — The woman that President Donald Trump has tapped to become the fifth member and final member of the Federal Communications Commission and cement the agency’s Republican majority mostly skated through her nomination hearing Wednesday morning…. Law360 is on it, so you are, too. A Law360 subscription puts you at the center of fast-moving legal issues, trends and developments so you can act with speed and confidence. Over 200 articles are published daily across more than 60 topics, industries, practice areas and jurisdictions. A Law360 subscription includes features such as Daily newsletters Expert analysis Mobile app Advanced search Judge information Real-time alerts 450K+ searchable archived articles And more! Experience Law360 today with a free 7-day trial. source

Trump's FCC Nominee Faces Light Scrutiny At Senate Hearing Read More »

Patch Tuesday: Microsoft Fixes 134 Vulnerabilities, Including 1 Zero-Day

Microsoft CEO Satya Nadella. Image: Microsoft News Microsoft’s Patch Tuesday security update for April included 134 flaws, one of which is an actively exploited zero-day flaw. The security patches for Windows 10 were unavailable when the Windows 11 patches were released. The Windows 10 patches have since arrived, but the delay was unusual. Tyler Reguly, associate director of security R&D at global cybersecurity software and services provider Fortra, suggested in an email to TechRepublic that the two separate releases and a 40-minute delay in the Windows 11 update might point to something unusual behind the scenes. SEE: What is Patch Tuesday? Microsoft’s Monthly Update Explained CVE-2025-29824 has been detected in the wild The zero-day vulnerability was CVE-2025-29824, an elevation of privilege bug in the Windows Common Log File System (CLFS) Driver. “This vulnerability is significant because it affects a core component of Windows, impacting a wide range of environments, including enterprise systems and critical infrastructure,” Mike Walters, president and co-founder of patch automation company Action1, wrote in an email. “If exploited, it allows privilege escalation to SYSTEM level—the highest privilege on a Windows system.” Elevation of privilege attacks require the threat actor to have a foothold in the system first. “Elevation of privilege flaws in CLFS have become especially popular among ransomware operators over the years,” Satnam Narang, Tenable’s senior staff research engineer, said in an email. “What makes this vulnerability particularly concerning is that Microsoft has confirmed active exploitation in the wild, yet at this time, no patch has been released for Windows 10 32-bit or 64-bit systems,” Ben McCarthy, lead cybersecurity engineer at security training company Immersive, added. “The lack of a patch leaves a critical gap in defense for a wide portion of the Windows ecosystem.” The delayed rollout of Windows 10 patches — paired with a 40-minute delay in the Windows 11 update — adds further weight to concerns about internal disruptions or challenges at Microsoft. While the reason for the delay remains unclear, security researchers are taking note of the timing, particularly given the active exploitation of CVE-2025-29824. CVE-2025-29824 has been exploited against “a small number of targets” in “organizations in the information technology (IT) and real estate sectors of the United States, the financial sector in Venezuela, a Spanish software company, and the retail sector in Saudi Arabia,” Microsoft disclosed. “I was recently discussing CLFS vulnerabilities and how they seem to come in waves,” Reguly noted. “When a vulnerability in CLFS is patched, people tend to dig around and look at what’s going on and come across other vulnerabilities in the process. If I was a gambler, I would bet on CLFS appearing again next month.” Remote code execution and Microsoft Office flaws are common patterns Other notable parts of April’s Patch Tuesday include a fix for CVE-2025-26663, a critical flaw that could affect organizations running Windows Lightweight Directory Access Protocol (LDAP) servers. Reguly highlighted CVE-2025-27472, a vulnerability in Mark of the Web (MOTW) that Microsoft listed as Exploitation More Likely.  “It is common to see MOTW vulnerabilities utilized by threat actors,” he said. “I wouldn’t be surprised if this is a vulnerability that we see exploited in the future.” SEE: Choose the right security applications for your business by balancing features, data storage, and cost.  Microsoft released multiple patches for CVEs in Office (CVE-2025-29791, CVE-2025-27749, CVE-2025-27748, and CVE-2025-27745). Microsoft Office’s popularity means these vulnerabilities have the potential for widespread problems, although they all require successful social engineering or remote code execution to inject a malicious file. While some of these CVEs enabled remote code execution (RCE), this month’s Patch Tuesday told a different story overall. Must-read security coverage “For the first time since August 2024, Patch Tuesday vulnerabilities skewed more towards elevation of privilege bugs, which accounted for over 40% (49) of all patched vulnerabilities,” Narang said. “We typically see remote code execution (RCE) flaws dominate Patch Tuesday releases, but only a quarter of flaws (31) were RCEs this month.” Reguly noted that Office, browsers, and MOTW have often appeared in Patch Tuesday updates lately. “If I were an infosec buyer, think CISO, I’d be looking at the trends in Microsoft vulnerabilities – recurring and commonly exploited technologies like Office, Edge, CLFS, and MOTW – and I’d be asking my vendors how they are helping me proactively defend against these types of vulnerabilities,” he said. Apple releases large security update As KrebsonSecurity pointed out, Apple users shouldn’t forget about security patches. Apple released a large security update on March 31, addressing some actively exploited vulnerabilities. In general, Patch Tuesday is a good time for organizations to push updates to company-owned devices. Consider backing up devices before updating in case something breaks in the newly installed software. source

Patch Tuesday: Microsoft Fixes 134 Vulnerabilities, Including 1 Zero-Day Read More »

NSO Hack Needed Apple's Calif. Servers, Foreign Journos Say

By Rachel Scharf ( April 10, 2025, 6:29 PM EDT) — Counsel for a group of El Salvador-based journalists urged the Ninth Circuit on Thursday to revive a lawsuit accusing Israeli spyware maker NSO Group of hacking their iPhones, saying the case belongs in California federal court because the alleged attacks relied on Apple’s servers within the Golden State…. Law360 is on it, so you are, too. A Law360 subscription puts you at the center of fast-moving legal issues, trends and developments so you can act with speed and confidence. Over 200 articles are published daily across more than 60 topics, industries, practice areas and jurisdictions. A Law360 subscription includes features such as Daily newsletters Expert analysis Mobile app Advanced search Judge information Real-time alerts 450K+ searchable archived articles And more! Experience Law360 today with a free 7-day trial. source

NSO Hack Needed Apple's Calif. Servers, Foreign Journos Say Read More »

DeepCoder delivers top coding performance in efficient 14B open model

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Researchers at Together AI and Agentica have released DeepCoder-14B, a new coding model that delivers impressive performance comparable to leading proprietary models like OpenAI’s o3-mini.  Built on top of DeepSeek-R1, this model gives more flexibility to integrate high-performance code generation and reasoning capabilities into real-world applications. Importantly, the teams have fully open-sourced the model, its training data, code, logs and system optimizations, which can help researchers improve their work and accelerate progress. Competitive coding capabilities in a smaller package The research team’s experiments show that DeepCoder-14B performs strongly across several challenging coding benchmarks, including LiveCodeBench (LCB), Codeforces and HumanEval+. “Our model demonstrates strong performance across all coding benchmarks… comparable to the performance of o3-mini (low) and o1,” the researchers write in a blog post that describes the model. Interestingly, despite being trained primarily on coding tasks, the model shows improved mathematical reasoning, scoring 73.8% on the AIME 2024 benchmark, a 4.1% improvement over its base model (DeepSeek-R1-Distill-Qwen-14B). This suggests that the reasoning skills developed through RL on code can be generalized effectively to other domains. Credit: Together AI The most striking aspect is achieving this level of performance with only 14 billion parameters. This makes DeepCoder significantly smaller and potentially more efficient to run than many frontier models. Innovations driving DeepCoder’s performance While developing the model, the researchers solved some of the key challenges in training coding models using reinforcement learning (RL). The first challenge was curating the training data. Reinforcement learning requires reliable reward signals indicating the model’s output is correct. As the researchers point out, “Unlike math—where abundant high-quality, verifiable data is readily available on the Internet—the coding domain suffers from a relative scarcity of such data.”  To address this problem, the DeepCoder team implemented a strict pipeline that gathers examples from different datasets and filters them for validity, complexity and duplication. This process yielded 24,000 high-quality problems, providing a solid foundation for effective RL training. The team also designed a straightforward reward function that only provides a positive signal if the generated code passes all sampled unit tests for the problem within a specific time limit. Combined with the high-quality training examples, this outcome-focused reward system prevents the model from learning tricks like printing memorized answers for public tests or optimizing for simple edge cases without solving the core problem. The model’s core training algorithm is based on Group Relative Policy Optimization (GRPO), a reinforcement learning algorithm that proved very successful in DeepSeek-R1. However, the team made several modifications to the algorithm to make it more stable and allow the model to continue improving as the training extends for a longer time. GRPO+ enables DeepCoder-14 to continue for longer durations without collapsing Credit: Together AI Finally, the team extended the model’s context window iteratively, first training it on shorter reasoning sequences and gradually increasing the length. They also developed a filtering method to avoid penalizing the model when it created reasoning chains that exceeded the context limits when solving a hard prompt.  DeepCoder was trained on 32K context problems but was also able to solve 64K tasks Credit: Together AI The researchers explain the core idea: “To preserve long-context reasoning while enabling efficient training, we incorporated overlong filtering… This technique masks out truncated sequences during training so that models aren’t penalized for generating thoughtful but lengthy outputs that exceed the current context limit.”  The training was gradually scaled from a 16K to a 32K context window, and the resulting model could also solve problems that required up to 64K tokens. Optimizing long-context RL training Training large models with RL, especially on tasks requiring long generated sequences like coding or complex reasoning, is computationally intensive and slow. A major bottleneck is the “sampling” step, where the model generates potentially thousands of tokens per example in the batch. Variations in response length mean some responses finish much later than others, leaving GPUs idle and slowing down the entire training loop.  To accelerate this, the team developed verl-pipeline, an optimized extension of the open-source verl library for reinforcement learning from human feedback (RLHF). The key innovation, which they call “One-Off Pipelining,” rearranges the response sampling and model updates to reduce the bottlenecks and accelerator idle time. One-Off Pipelining Their experiments showed that one-off pipelining provided up to a 2x speedup for coding RL tasks compared to baseline implementations. This optimization was crucial for training DeepCoder within a reasonable timeframe (2.5 weeks on 32 H100s) and is now open-sourced as part of verl-pipeline for the community to use and build upon.  Enterprise impact The researchers have made all the artifacts for training and running DeepCoder-14B available on GitHub and Hugging Face under a permissive license. “By fully sharing our dataset, code, and training recipe, we empower the community to reproduce our work and make RL training accessible to all,” the researchers write. DeepCoder-14B powerfully illustrates a broader, accelerating trend in the AI landscape: the rise of highly capable yet efficient and openly accessible models.  For the enterprise world, this shift signifies more options and higher accessibility of advanced models. Cutting-edge performance is no longer solely the domain of hyperscalers or those willing to pay premium API fees. Models like DeepCoder can empower organizations of all sizes to leverage sophisticated code generation and reasoning, customize solutions to their specific needs, and securely deploy them within their environments.  This trend can lower the barrier to entry for AI adoption and foster a more competitive and innovative ecosystem, where progress is driven through open source collaboration. source

DeepCoder delivers top coding performance in efficient 14B open model Read More »

Anthropic’s and OpenAI’s new AI education initiatives offer hope for enterprise knowledge retention

These education platforms essentially serve as testing grounds for more responsible AI implementation models that could transfer to enterprise settings. The Socratic questioning in Anthropic’s Learning Mode and OpenAI’s focus on deeper engagement suggest a fundamental shift in AI design philosophy, from tools that provide answers to tools that enhance human capabilities. “To counter cognitive atrophy, organizations must design for active engagement: Teams should be encouraged to interrogate AI outputs, not just accept them. Think ‘copilot,’ not ‘autopilot,’“ Sengar said. He suggested that businesses adopt techniques inspired by AI-driven education models. These include Socratic prompting that encourages critical thinking, progressive disclosure of information rather than immediate answers, and requiring decision rationales to ensure human judgment remains sharp. Each of these approaches mirrors what’s being pioneered in educational settings but with adaptation for workplace contexts where preserving institutional knowledge is particularly crucial. source

Anthropic’s and OpenAI’s new AI education initiatives offer hope for enterprise knowledge retention Read More »

What’s inside the LLM? Ai2 OLMoTrace will ‘trace’ the source

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Understanding precisely how the output of a large language model (LLM) matches with training data has long been a mystery and a challenge for enterprise IT. A new open-source effort launched this week by the Allen Institute for AI (Ai2) aims to help solve that challenge by tracing LLM output to training inputs. The OLMoTrace tool allows users to trace language model outputs directly back to the original training data, addressing one of the most significant barriers to enterprise AI adoption: the lack of transparency in how AI systems make decisions. OLMo is an acronym for Open Language Model, which is also the name of Ai2’s family of open-source LLMs. On the company’s Ai2 Playground site, users can try out OLMoTrace with the recently released OLMo 2 32B model. The open-source code is also available on GitHub and is freely available for anyone to use. Unlike existing approaches focusing on confidence scores or retrieval-augmented generation, OLMoTrace offers a direct window into the relationship between model outputs and the multi-billion-token training datasets that shaped them. “Our goal is to help users understand why language models generate the responses they do,” Jiacheng Liu, researcher at Ai2 told VentureBeat. How OLMoTrace works: More than just citations LLMs with web search functionality, like Perplexity or ChatGPT Search, can provide source citations. However, those citations are fundamentally different from what OLMoTrace does. Liu explained that Perplexity and ChatGPT Search use retrieval-augmented generation (RAG). With RAG, the purpose is to improve the quality of model generation by providing more sources than what the model was trained on. OLMoTrace is different because it traces the output from the model itself without any RAG or external document sources. The technology identifies long, unique text sequences in model outputs and matches them with specific documents from the training corpus. When a match is found, OLMoTrace highlights the relevant text and provides links to the original source material, allowing users to see exactly where and how the model learned the information it’s using. Beyond confidence scores: Tangible evidence of AI decision-making By design, LLMs generate outputs based on model weights that help to provide a confidence score. The basic idea is that the higher the confidence score, the more accurate the output. In Liu’s view, confidence scores are fundamentally flawed.  “Models can be overconfident of the stuff they generate and if you ask them to generate a score, it’s usually inflated,” Liu said. “That’s what academics call a calibration error—the confidence that models output does not always reflect how accurate their responses really are.” Instead of another potentially misleading score, OLMoTrace provides direct evidence of the model’s learning source, enabling users to make their own informed judgments. “What OLMoTrace does is showing you the matches between model outputs and the training documents,” Liu explained. “Through the interface, you can directly see where the matching points are and how the model outputs coincide with the training documents.” How OLMoTrace compares to other transparency approaches Ai2 is not alone in the quest to better understand how LLMs generate output. Anthropic recently released its own research into the issue. That research focused on model internal operations, rather than understanding data. “We are taking a different approach from them,” Liu said. “We are directly tracing into the model behavior, into their training data, as opposed to tracing things into the model neurons, internal circuits, that kind of thing.” This approach makes OLMoTrace more immediately useful for enterprise applications, as it doesn’t require deep expertise in neural network architecture to interpret the results. Enterprise AI applications: From regulatory compliance to model debugging For enterprises deploying AI in regulated industries like healthcare, finance, or legal services, OLMoTrace offers significant advantages over existing black-box systems. “We think OLMoTrace will help enterprise and business users to better understand what is used in the training of models so that they can be more confident when they want to build on top of them,” Liu said. “This can help increase the transparency and trust between them of their models, and also for customers of their model behaviors.” The technology enables several critical capabilities for enterprise AI teams: Fact-checking model outputs against original sources Understanding the origins of hallucinations Improving model debugging by identifying problematic patterns Enhancing regulatory compliance through data traceability Building trust with stakeholders through increased transparency The Ai2 team has already used OLMoTrace to identify and correct their models’ issues. “We are already using it to improve our training data,” Liu reveals. “When we built OLMo 2 and we started our training, through OLMoTrace, we found out that actually some of the post-training data was not good.” What this means for enterprise AI adoption For enterprises looking to lead the way in AI adoption, OLMoTrace represents a significant step toward more accountable enterprise AI systems. The technology is available under an Apache 2.0 open-source license, which means that any organization with access to its model’s training data can implement similar tracing capabilities. “OLMoTrace can work on any model, as long as you have the training data of the model,” Liu notes. “For fully open models where everyone has access to the model’s training data, anyone can set up OLMoTrace for that model and for proprietary models, maybe some providers don’t want to release their data, they can also do this OLMoTrace internally.” As AI governance frameworks continue to evolve globally, tools like OLMoTrace that enable verification and auditability will likely become essential components of enterprise AI stacks, particularly in regulated industries where algorithmic transparency is increasingly mandated. For technical decision-makers weighing the benefits and risks of AI adoption, OLMoTrace offers a practical path to implementing more trustworthy and explainable AI systems without sacrificing the power of large language models. source

What’s inside the LLM? Ai2 OLMoTrace will ‘trace’ the source Read More »

Qualtrics X4 Highlights: AI-Powered Research Is Expanding

At X4 2025, Qualtrics announced several new features, all supported by AI. Two highlights were Experience Agents and Qualtrics Edge. Qualtrics Edge is a platform for research and insights professionals that combines AI, synthetic insights, and market research data to provide visibility into competitors and adjacent markets. The Edge Audiences feature allows users to access fully human, fully synthetic, or hybrid panels, which is a useful companion to support early phases of discovery work and generating hypotheses. These are advancements that can streamline customer and experience research and broaden customer understanding, and using synthetic insights in addition to real human insights that are gained through traditional user research methods is a big step. We’ll see more AI-powered solutions like these emerge in the research space with advancements in AI. What do advancements like these mean for experience research today and in the future? Here are my three takeaways: AI-supported research is real and serious. A year ago, for many researchers, the idea of using AI to conduct research was an interesting nice-to-have, and for some, it was out of the question due to concerns about privacy and biased data. But today, AI-supported research is real, and teams are actively seeking ways to understand the use cases and learn how to use AI in their research practices. Researchers’ expectations are not fully met. AI-powered features to improve the research flow, such as AI-generated transcriptions and insight summaries, are now nearly table stakes. In many cases, however, researchers don’t find these features nuanced enough. They need more depth in the analysis, more guidance in decision-making, and more support in data visualization and storytelling, and these expectations will continue to increase. Synthetic insights are still not a replacement for research with humans. Synthetic audiences are promising for early phases of experience research — such as discovery work and generating hypotheses — especially when you’re targeting niche audiences or when recruitment is difficult. But synthetic data is still not a replacement for research with real humans. Determining when to use synthetic insights and how to supplement research with real humans still requires research expertise, and success still depends on the ability to ask the right research questions and determine the right research problem. If you are a Forrester client and would like to discuss this topic further or have questions about the experience research landscape, set up a conversation with me.  source

Qualtrics X4 Highlights: AI-Powered Research Is Expanding Read More »