# Why AI Resolution Rate Claims on the Shopify App Store Rarely Hold Up > The AI resolution rate promised by Shopify apps is often a vanity metric that hides unpredictable costs and a significant gap between answering questions and solving problems. Source: https://arbyn.app/blog/why-ai-resolution-rate-claims-on-the-shopify-app-store-rarely-hold-up Published: 2026-07-22 --- It’s the first Monday of the month, a familiar and often dreaded ritual for any Shopify store owner. You navigate to the billing dashboard for your helpdesk software, bracing yourself as you expect the usual, predictable charge for your handful of dedicated support agents. Instead of clarity, you see a number that’s 40% higher than last month, maybe even double, instantly turning a manageable $500 software cost into a volatile and alarming $900 expense. The culprit is a line item that wasn’t there a few quarters ago: “AI Resolutions.” This is especially jarring because your app’s dashboard proudly boasts a 60% or 70% AI resolution rate, a figure that promised significant cost savings and operational efficiency when you first signed up. The number on the invoice, however, tells a different, more painful story of financial unpredictability. This growing disconnect between a high advertised AI resolution rate on the Shopify App Store and the escalating monthly bill isn't an accident; it’s a direct consequence of a broken, misleading metric that has become the default sales pitch for an entire category of AI tools. That percentage doesn’t reflect a genuine reduction in your team's workload or a measurable improvement in your customer experience. It reflects a billing model deliberately designed to penalize you for the very volume it was supposed to manage, turning your store's growth into a financial liability. The Deceptive Simplicity of "Resolution Rate" On the surface, the AI resolution rate seems like the perfect yardstick for an automated support tool, a single, digestible number to gauge effectiveness. It’s a clean percentage that promises to quantify an AI’s impact: the portion of customer inquiries it successfully handles from start to finish without any human intervention. App developers plaster figures like "70% resolution" or "80% of tickets automated" across their Shopify App Store listings and homepages, framing it as the primary indicator of value and ROI. The appeal is undeniably potent, especially when facing industry data showing that 30-50% of all support tickets are simple "Where is my order?" (WISMO) questions. The metric suggests a utopian future where this relentless tide of repetitive inquiries, return requests, and basic product questions is managed automatically around the clock, freeing you and your team to focus on high-value activities like marketing, product development, and growing the business. A high rate implies immediate cost savings, 24/7 availability for a global customer base, and the instant gratification of answers for your customers. It feels like the ultimate lever for achieving operational efficiency and scaling gracefully. The core problem is that the definition of a "resolution" is dangerously pliable, especially when a per-incident billing model is attached to it. What truly counts as a resolution? If a customer asks for their tracking number, the AI provides it, and the customer closes the chat window, is that a success? Most platforms, including Intercom, would say yes, logging it as either a "confirmed" or, more frequently, an "assumed" resolution. But what if the customer's real, unstated problem was that the package was stalled in transit for ten days and they actually needed to request a reshipment, but they gave up when the AI couldn't understand that nuance? The ticket is still marked "resolved," and in many cases, billed. This fundamental ambiguity is where the entire metric begins to unravel and mislead. Many platforms count any interaction that doesn't end in an explicit, user-triggered escalation to a human agent as a successful resolution. This includes conversations where the customer simply stops replying out of pure frustration or because they found the answer on their own elsewhere. These "assumed resolutions" inflate the percentage but often represent a catastrophic failure in customer experience, not a success. A customer who abandons a chat is not a happy customer; they are a customer at high risk of churning, especially when research consistently shows that around a third of consumers will walk away from a brand they love after just one bad service experience. This fundamental definitional flaw is compounded by the fact that the metric treats all inquiries as equal in value and complexity. An AI that resolves a thousand simple WISMO questions is algorithmically seen as more effective than one that resolves a hundred complex pre-sale inquiries that lead to high-value orders. Imagine those thousand WISMO tickets are from past, low-value customers, while the hundred pre-sale questions are about product compatibility or sizing and could collectively generate $10,000 in new revenue. The resolution rate metric doesn’t distinguish between a low-value, one-touch answer and a high-value, multi-step problem solved. As one of Zendesk's own guides on customer service metrics implies, a high first-contact resolution rate can easily mask underlying problems like inaccurate answers, a frustrating user experience, or a failure to address the customer's actual issue. The percentage becomes a pure vanity metric, optimized for marketing claims rather than genuine business impact or customer satisfaction. It creates a powerful illusion of performance that completely falls apart under the financial scrutiny of your monthly invoice, where every one of those thousand "resolutions" often carries a discrete, and sometimes hidden, cost. How Per-Resolution Billing Turns Success into a Penalty The most damaging aspect of the AI resolution rate isn't the metric itself, but the punitive billing model it enables. Most of the major AI helpdesks targeting the Shopify ecosystem, including popular choices like Gorgias, Intercom, and Zendesk, have adopted per-resolution pricing. At first glance, the model sounds equitable: you only pay when the AI successfully solves a problem. Intercom’s AI model is built around its widely advertised $0.99 per resolution fee. Gorgias charges a per-resolution fee on top of its base plan's ticket allowance, a confusing structure that can lead to double-billing for a single automated conversation. Zendesk’s model is even more complex and costly, layering a per-resolution fee on top of mandatory per-agent seat licenses and a separate AI add-on that can run an additional $50 per agent per month. These costs are not only high but are designed to be opaque, making true cost prediction nearly impossible for a growing business. This model creates a direct and unavoidable conflict of interest between you and your software vendor. Your primary goal is to resolve as many customer issues as efficiently as possible to reduce operational costs and improve satisfaction. The vendor's goal, driven by their revenue model, is to maximize the number of billable resolutions their system logs. The better their AI performs by their own loose definition, the higher your bill becomes. Imagine your store generates 2,000 support conversations in a month. If your chosen AI tool achieves the advertised 60% resolution rate, you are now paying for 1,200 individual resolutions. On the Gorgias Pro plan, 190 of those are included and the remaining 1,010 cost $1.50 each, which is $1,515 in AI fees alone, plus your base plan cost and the cost of those 1,200 helpdesk tickets being counted against your plan's limit. On Intercom, that's nearly $1,200 (1,200 x $0.99). On Zendesk you cannot run this calculation at all, because they publish no per-resolution rate, and that is before you even account for your mandatory seat licenses or the Copilot add-on. Suddenly, the efficiency you were promised comes with a variable, uncapped, and dangerously unpredictable monthly cost. A spike in traffic from a successful marketing campaign or a viral social media post becomes a financial liability, as each new customer question adds directly to a metered bill. This "growth trap" is the central, undeniable failure of the per-resolution model. Flat-rate pricing models, in stark contrast, offer predictability, which is a cornerstone of sound financial planning for any business, from a startup to an enterprise. You pay one fixed monthly fee for a certain number of conversations or, in the best-case scenario, for unlimited volume. The table below illustrates the stark difference in cost predictability and financial risk. While a per-resolution model might seem cheaper at a very low, almost negligible volume, it quickly becomes exponentially more expensive and volatile as your store grows. A flat-rate model aligns the vendor's success with yours; they have a powerful incentive to handle your increasing volume as efficiently as possible to protect their own margins, while your costs remain completely fixed and predictable. This financial alignment is critical for a healthy partnership. You should be rewarded for scaling your business, not penalized for it with a support bill that grows in lockstep with your success. Billing Model How It Works Cost at 2,000 Conversations/Month (60% AI Resolution) Predictability Per-Resolution (e.g., Intercom, Gorgias) A base subscription fee plus $1.50 for each automated interaction past the allowance in your plan. That fee can be charged on top of the ticket being counted against your plan's limit. ~$1,080 - $2,220 + base plan fees + potential overages. Low. Bill scales with volume and AI performance. Per-Agent + Per-Resolution (e.g., Zendesk) A fee for each human agent seat, plus a Copilot add-on at $50/agent/month paid yearly, plus a fee for each AI resolution that Zendesk does not publish. ~$2,400+ (for a small team, including seats, add-ons, and resolutions). Very Low. Scales with headcount, volume, and AI usage. Flat Rate (e.g., Arbyn) One fixed price for unlimited conversations and unlimited AI resolutions. $99. High. Costs are fixed regardless of volume. The Action Gap: When "Resolved" Fails to Solve the Real Problem Even if you could set aside the punitive and unpredictable billing models, a high AI resolution rate often conceals a critical capability gap between answering and acting. Most AI tools currently on the market are excellent at functioning as question-answering machines. They operate as sophisticated knowledge retrieval systems, pulling information from your help center articles, order data via an API, and your product catalog to respond to customer queries. They can instantly tell a customer where their order is, explain the nuances of your return policy, or list a product's material specifications. This is useful for deflecting the most common, repetitive questions that plague support teams, which consistently make up that 30-50% of total support volume. However, this capability falls dramatically short of true, end-to-end resolution, because customer problems often require more than just an answer; they require a specific action to be performed within your Shopify store's backend or other integrated systems. Consider the natural, multi-step evolution of a common customer conversation. A "Where is my order?" query isn't just a request for a tracking number 90% of the time. It is often the prelude to a more complex and urgent problem: the package is lost, it was delivered to the wrong address, the contents were damaged, or the customer needs to change the delivery instructions. An AI that can only provide the tracking link has not solved the underlying issue; it has merely completed the first, simplest step. When the customer inevitably replies, "The tracking says it was delivered, but I don't have it," the AI hits a functional wall. The conversation must then be escalated to a human agent, who performs a long sequence of manual tasks: looking up the order in Shopify, cross-referencing the tracking with the carrier's portal, communicating with the warehouse, and finally initiating a reshipment or refund. The initial AI "resolution" was hollow, and the process ultimately created more work and a disjointed, frustrating experience for the customer. The AI handled the part that was already easy, leaving the actual value-added work for your team while still potentially logging a billable event. True resolution requires an AI that is not just a conversationalist but a genuine store owner within your tech stack. It needs the authenticated permissions and deep integrations to perform the same actions that a human agent would to fully solve a problem. This means directly changing a shipping address in Shopify for an unfulfilled order, canceling an order and triggering the right warehouse notifications, issuing a precise refund for a specific line item while keeping the rest of the order active, applying a unique, single-use discount code to a pending checkout to save a sale, or creating a new draft order for a complex reshipment. Without this fundamental ability to execute tasks, the AI is perpetually stuck at the surface level of customer service. The "resolution rate" in this context merely measures the AI's ability to handle trivial, informational queries, not its ability to manage the full lifecycle of a customer issue. Store owners are left paying a premium for an AI that only deflects the simplest questions while their team remains burdened with the actual, time-consuming work of solving the problems that determine whether a customer returns to buy again or churns forever. Looking Beyond the Rate: A Framework for Evaluating Shopify AI Escaping the pervasive trap of the AI resolution rate requires a fundamental shift in how you evaluate and procure support automation tools. Instead of anchoring your decision on a single, easily manipulated percentage, you need a robust framework that assesses an AI's true, measurable business value. This means prioritizing financial predictability, deep functional capability, and complete operational control above all else. The goal is not to find the tool with the highest advertised resolution rate, but to find the one that offers a transparent and scalable cost structure while genuinely reducing the manual workload on your team. It’s a strategic move from chasing a vanity metric to investing in a sustainable operational asset that provides a clear and predictable return on investment. This requires looking past the flashy marketing on the Shopify App Store and digging deep into the core mechanics of the software's architecture and its business model. Start by scrutinizing the billing model with extreme prejudice, as it is the single greatest indicator of future pain or success. Is it a flat, predictable monthly fee, or does it scale with usage in a way that creates volatility? A per-resolution or per-ticket model is a major red flag, indicating that your costs will be variable and will increase directly with your business growth, creating a clear conflict of interest. Demand a pricing structure that you can forecast and budget for with confidence, one that doesn't punish you for a successful sales month. As one analysis of SaaS pricing notes, a key benefit of subscription models is the ability to "forecast their income more accurately, which aids in financial planning, resource allocation, and investment decisions", a benefit that should apply to the customer just as much as the vendor. A flat-rate plan for unlimited conversations provides the highest degree of cost certainty, making it a much safer and more strategic bet for any scaling Shopify store. This financial transparency is the first and most important pillar of a sound evaluation. As the old business adage goes, "Predictability beats cheapness, nine times out of ten." Next, you must rigorously audit the AI's actual capabilities, moving far beyond the marketing claims. Ask for a live, screen-shared demonstration of the specific actions it can perform directly within a real Shopify instance. Can it process a return for a damaged item and simultaneously tag the customer for a follow-up? Can it modify an order's shipping address post-purchase and leave an internal note for the fulfillment team? Can it issue a partial refund for a single item in a multi-item order without affecting the rest of the purchase? Can it create and apply a unique, single-use discount code for a specific customer who had a poor experience? An AI that can only answer questions is a chatbot; an AI that can execute multi-step tasks is an agent. This distinction is crucial and represents the difference between marginal and transformative value. The latter provides a far greater return on investment by automating entire workflows, not just the initial customer response. Finally, assess the degree of control you have over the system. You must be able to define and customize escalation paths, set granular rules for when a human should always intervene, and customize the AI's tone, language, and policies to perfectly match your brand's identity. An AI should be a powerful tool that you command, not a black box that operates on its own terms and leaves you with a surprise bill. Stop Chasing Percentages and Start Demanding Predictability The industry-wide obsession with the AI resolution rate has led countless Shopify store owners down a painful path of unpredictable costs, frustrating customer experiences, and underwhelming results. This single metric, while seductive in its simplicity, has been co-opted by billing models that actively punish growth and obscure the true cost of support automation. A high percentage means absolutely nothing if it’s built on a foundation of hollow definitions of "resolved", such as a customer simply giving up in frustration, and results in a volatile monthly invoice that systematically eats away at your hard-earned margins. The intense focus on this number has created a deep and damaging misalignment between what store owners need for sustainable, profitable growth and what many AI tools are optimized to deliver for their own quarterly revenue targets. This isn't just a bad business practice; it's a structural flaw in how value is currently measured and sold in the e-commerce support space. The path forward is to consciously reject the vanity metric in favor of tangible, predictable value. This journey starts with demanding absolute predictability and transparency in pricing. A flat-rate billing model is not just a feature; it is a strategic necessity for any business that plans to scale its operations. It fundamentally transforms your support operation from a variable cost center into a fixed, predictable investment. By choosing a platform like Arbyn, which offers unlimited conversations for a single flat fee of $99 per month, you are permanently insulated from the "growth trap" of per-resolution fees. Imagine a TikTok video goes viral, tripling your order volume and support inquiries overnight from 1,000 to 3,000. On a per-resolution model, this success would come with a surprise bill for thousands of dollars. With a flat rate, it becomes a pure win for your top line, not a source of financial panic about your next helpdesk bill. Whether you handle 500 conversations or 50,000, your cost remains the same, allowing you to budget with certainty and reinvest your profits directly into further growth. This essential financial sanity must be paired with genuine, action-oriented capability. An AI's true worth is not measured by the number of questions it can answer, but by the number of problems it can solve from end to end. Arbyn is designed from the ground up to be an agent, not just a chatbot. It is built to execute the very actions that cause escalations, manual work, and customer frustration on other platforms. While critical money-moving actions like refunds and cancellations are rightly gated by your one-click approval for complete financial control, a crucial safeguard against errors, Arbyn queues up and executes these tasks for you, along with autonomously handling address changes, order modifications, and more. It closes the loop, moving beyond simple conversation to decisive action. The choice for store owners is becoming increasingly clear: you can continue to chase an inflated percentage that leads to unpredictable bills and operational headaches, or you can anchor your operations on the financial certainty and functional power of a flat-rate platform that actually gets the real work done. Your bottom line will thank you for it. --- ## Pricing - **Arbyn Starter** - $0/month, permanently free. 150 conversations / month. Resets 1st of each month. - **Arbyn Agent** - $99/month flat, unlimited conversations. Or $990/year (2 months free, saves $198, 17% off). - **There is no trial.** Billing starts immediately on the Agent plan. The free Starter plan is permanent. - The conversation cap is the only difference between plans. There is no feature gating. ## Channels Live today: **support email** and **on-site live chat**. That is the complete list. SMS, Instagram DMs, Facebook Messenger, WhatsApp and Voice are on the roadmap and are NOT live. Arbyn does not edit orders or change line items. Money-moving actions (cancel, refund, discount, gift card, reship, return) require the store owner's approval, and then Arbyn performs them. Running them fully autonomously is a beta authorization and is in development. Shipping address changes are already autonomous. ## What Arbyn does on a Shopify order - **Change the shipping address**: Live. Arbyn does this on its own. Arbyn updates the shipping address on the Shopify order itself, inside the conversation, and writes the change to the order timeline. - **Cancel an order**: Live. You approve it, then Arbyn cancels the order. Anything that moves money waits for the store owner's approval. That is a deliberate control, not a missing feature. Once you approve, Arbyn fires Shopify's order cancellation itself and confirms it to the customer. - **Issue a refund**: Live. You approve it, then Arbyn issues the refund. Arbyn prepares the refund against the original payment method and sends it to you. On approval it files the refund in Shopify. You can cap the value it is allowed to prepare, per channel. - **Apply a discount**: Live. Arbyn creates a real Shopify discount and applies it to the cart, handing the shopper a checkout with the code already on it. It can also issue a discount code on an order once you approve it. - **Send a gift card, or reship an order**: Live. You approve it, then Arbyn does it. Arbyn creates the gift card, or raises the replacement order, in Shopify once you approve. - **Start a return**: Live. You approve it, then Arbyn opens the return. Arbyn opens the return in Shopify on your approval. - **Look up a gift card or store-credit balance**: Live. Arbyn does this on its own. "Do I have store credit left?" is a question most support tools answer with a human. Arbyn reads the balance itself, for a verified customer or from the code they give you, and reports the masked card, the balance and the expiry. If there is no card, it says so rather than guessing. - **Handle a subscription question**: Live. You choose what it does. Arbyn knows which of your products are sold as a subscription, shows that on the product card in the conversation, and sends a subscriber to their subscription management page to pause, skip or cancel. It answers how your subscriptions work from your own knowledge, but it does not read an individual customer's contract, so it will not state their renewal date or status. Most cancels are a customer with product piling up, and the fix is getting them to the page where they can slow the cadence down. Reading the contract itself is on the roadmap. - **Answer support email and live chat**: Live. Arbyn reads every inbound support email and every chat, works out the intent, pulls the live Shopify context, and replies in your brand voice. Money-moving actions (cancel, refund, discount, gift card, reship, return) require the store owner's approval, and then Arbyn performs them. Running them fully autonomously is a beta authorization and is in development. Shipping address changes are already autonomous.