Professional Services Contract with CaremarkPCS Health for pharmacy benefit management services

All five models reached this verdict. Section 2.5(B) on page 11 has the vendor accept a fiduciary role only for initial claim adjudication and appeals, and then states that for everything else it is not the plan administrator, not a named fiduciary, has no discretionary authority and is not deemed a fiduciary under ERISA or state law. Formulary placement, MAC pricing and manufacturer money therefore sit outside any duty of loyalty to the city. The model contract published by Mark Cuban and collaborators requires loyalty and care owed solely to the plan across the whole relationship, and page 10 confirms the vendor keeps manufacturer fees, so this is the opposite of the model.

Model votes: Kimi K3 bad, Grok 4.6 bad, GPT 5.6 Sol bad, Claude Opus 5 bad, Gemini 3.1 Pro bad

Three of five models read this as partial; Gemini 3.1 Pro called it missing because no clause requires a full compensation inventory or an officer certification, and Kimi K3 called it bad because the disclosure mainly authorizes retention. The disclosure of manufacturer fees on pages 9 and 10 does tell the city that the vendor and its affiliates take manufacturer administrative fees of one to four percent of wholesale acquisition cost across the book of business, plus purchase discounts at affiliated pharmacies, and section 21.3 on page 28 warrants that no contingent fee was paid to win the contract. Nothing requires the amounts actually earned on city claims to be reported, no affiliate ownership touching claims is listed, and no named officer certifies quarterly or annually. The subject is addressed, so the majority verdict of partial stands, but it is far weaker than the model.

Model votes: Kimi K3 bad, Grok 4.6 partial, GPT 5.6 Sol partial, Claude Opus 5 partial, Gemini 3.1 Pro missing

Three of five models read this as missing; Claude Opus 5 and GPT 5.6 Sol called it partial because section 14.3 on page 21 requires sixty days notice of a change in ownership interest greater than fifty percent or a change of control. That notice clause is real, but it is not an exhibit listing owners, parents, subsidiaries, affiliates, group purchasing organizations, rebate aggregators or mail and specialty pharmacies, and nothing requires a refreshed list within thirty days of a change. The exhibit list on page 32 names performance guarantees, fees, the business associate agreement, SBEDA, the request for proposal and the proposal, not an ownership schedule, and the conflict of interest article on pages 27 and 28 concerns city officers and employees. The contract refers to affiliates and vendor owned pharmacies repeatedly without naming any of them, so the majority verdict stands.

No verbatim quote is available for this clause.

Model votes: Kimi K3 missing, Grok 4.6 missing, GPT 5.6 Sol partial, Claude Opus 5 partial, Gemini 3.1 Pro missing

All five models reached this verdict. Section 1.12 on page 2 defines Rebates by label as formulary rebates, base and market share, collected as a group purchasing organization on brand drugs. Page 10 then removes fees, other compensation and concurrent or retrospective purchase discounts from the term and assigns them exclusively to the vendor or its affiliates. Brand and generic status are whatever the vendor identifies from Medi-Span on pages 1 and 2, the MAC list is a price the vendor establishes and amends, and specialty on page 3 is what the vendor uses. These are the relabeling loopholes that the definitions section of the model contract published by Mark Cuban and collaborators is written to close.

Model votes: Kimi K3 bad, Grok 4.6 bad, GPT 5.6 Sol bad, Claude Opus 5 bad, Gemini 3.1 Pro bad

All five models reached this verdict. Page 9 promises to remit the Rebates received on city claims each quarter under Exhibit B, and Exhibit B is not in the posted document, so neither the share nor the timing can be read. What can be read is decisive on its own: page 10 lets the vendor and its affiliates receive and retain manufacturer administrative fees of one to four percent of wholesale acquisition cost and affiliate purchase discounts, and declares that this money belongs exclusively to the vendor and is not a Rebate. Page 9 also waives any interest or time value on rebate money and lets the vendor delay remittance on termination. That contradicts the one hundred percent pass through of all manufacturer revenue in the model contract, whatever Exhibit B says about the defined Rebates.

Model votes: Kimi K3 bad, Grok 4.6 bad, GPT 5.6 Sol bad, Claude Opus 5 bad, Gemini 3.1 Pro bad

Three of five models read this as bad; Gemini 3.1 Pro called it missing because no clause sets acquisition cost pricing, and Kimi K3 called it unclear because the rates would sit in the withheld Exhibit B. The pages are readable and the body text addresses the subject, so the unclear vote is overruled under rule 3 and the majority stands. Page 4 runs claims at the vendor's own mail and specialty pharmacies through the same lowest of discounted rate, usual and customary and MAC formula as retail, and page 10 states that purchase discounts on products bought by vendor affiliated dispensing pharmacies belong exclusively to the vendor or its affiliates. That is a retained affiliate margin in plain words, the harm the acquisition cost clause in the model contract published by Mark Cuban and collaborators exists to remove. Page 18 adds that the city funds payments to the vendor's affiliated pharmacies.

Model votes: Kimi K3 unclear, Grok 4.6 bad, GPT 5.6 Sol bad, Claude Opus 5 bad, Gemini 3.1 Pro missing

Three of five models read this as partial; Gemini 3.1 Pro and Kimi K3 called it missing because a lesser of clause is not a published external benchmark. Page 4 does give a claim by claim test: the ingredient cost is the lowest of the discounted rate, the pharmacy's reported usual and customary price and the vendor MAC rate. That stops the plan paying more than the pharmacy's own shelf price at a network pharmacy, which is a ceiling of a kind and is checked on every claim rather than on average. It is not the model standard because the benchmark is the pharmacy's self reported price rather than a transparent published cash price, and mail and specialty claims run on rates in the withheld Exhibit B with no stated ceiling. The majority verdict stands.

Model votes: Kimi K3 missing, Grok 4.6 partial, GPT 5.6 Sol partial, Claude Opus 5 partial, Gemini 3.1 Pro missing

Three of five models read this as partial; Gemini 3.1 Pro and Kimi K3 called it missing because most of what the item asks for is absent. Pages 4 and 5 require network pharmacies to collect from the member the lowest of the copayment, the discounted price, the vendor MAC rate or the pharmacy's usual and customary price, so a member should not pay more at the counter than the pharmacy's own cash price. That is the first half of the model clause. There is no promise that a cash purchase made outside the benefit counts toward the deductible or out of pocket maximum, and accumulator adjustments are not mentioned anywhere in the text. The subject is addressed in part, so the majority verdict of partial stands with reduced confidence.

Model votes: Kimi K3 missing, Grok 4.6 partial, GPT 5.6 Sol partial, Claude Opus 5 partial, Gemini 3.1 Pro missing

Four of five models read this as bad; Claude Opus 5 called it partial because the city receives a copy of the list before signing and on reasonable request. Section 1.9 on page 2 makes MAC a unit price the vendor establishes and amends from time to time, on a list that is specific to the city and expressly not the federal list. There is no update schedule, no pharmacy appeal path with deadlines, and nothing says the list used to bill the city is the same list used to pay pharmacies. A vendor controlled, client specific MAC list is the mechanism the model contract published by Mark Cuban and collaborators names as the classic spread tool, so the majority verdict stands.

Model votes: Kimi K3 bad, Grok 4.6 bad, GPT 5.6 Sol bad, Claude Opus 5 partial, Gemini 3.1 Pro bad

Two of five models read this as bad, two as missing, and Kimi K3 voted unclear because the discount rates sit in the withheld Exhibit B. The pages that decide the question are readable, so the unclear vote is overruled under rule 3, leaving a tie between bad and missing that the rule sends back to the text. On re-read, the contract is not merely silent: page 4 defines what the city pays as a vendor set discounted rate, the pharmacy's usual and customary price or the vendor MAC rate, page 2 makes that MAC a list applicable to the city that the vendor maintains for its generic pricing program, page 5 says pharmacies are paid under their own network agreements with the vendor, and page 15 bars the city from auditing those agreements. The city's price and the pharmacy's payment are set on separate tracks and the contract blocks the one check that would reveal a difference, which permits retained spread in every channel. Confidence is low because no sentence says the vendor keeps the difference.

Model votes: Kimi K3 unclear, Grok 4.6 bad, GPT 5.6 Sol missing, Claude Opus 5 bad, Gemini 3.1 Pro missing

All five models reached this verdict. Page 9 has the city authorize the vendor to contract with drug makers as a group purchasing organization for the plan, and page 10 then assigns the fees, compensation and concurrent or retrospective purchase discounts collected in that role exclusively to the vendor or its affiliates. The non interference clause on page 10 also bars the city from seeking rebates or discounts from any manufacturer or third party on its own for the whole term and calls a breach material. The model contract published by Mark Cuban and collaborators sends purchasing entity revenue to the plan on disclosed drivers; here it is kept.

Model votes: Kimi K3 bad, Grok 4.6 bad, GPT 5.6 Sol bad, Claude Opus 5 bad, Gemini 3.1 Pro bad

All five models reached this verdict. The city adopted the vendor's own Performance Drug List as its formulary, and section 1.10 on page 2 says that list is created, maintained and amended by the vendor and approved by the vendor's own committee. Page 8 gives the city thirty days notice before a drug is added, removed or moved between tiers, and page 9 gives quarterly notice of changes to the specialty formulary, but notice is not approval and the city has no right to reject a change or to receive a lowest net cost analysis. The party that collects the rebates decides which drugs are preferred, which is what the formulary section of the model contract published by Mark Cuban and collaborators exists to prevent.

Model votes: Kimi K3 bad, Grok 4.6 bad, GPT 5.6 Sol bad, Claude Opus 5 bad, Gemini 3.1 Pro bad

Four of five models read this as missing; GPT 5.6 Sol called it partial because page 8 lists cost effectiveness among the reasons the vendor may change the drug list. That list on page 8 is a menu of things the vendor may consider, among other things, and it binds the vendor to nothing for this plan. There is no lowest net cost standard, no annual comparison of the city's results against the vendor's book of business, and no duty to disclose when a drug sits above a cheaper equivalent. The performance standards on pages 13 and 14 measure service accuracy and the audit article on pages 14 to 17 covers claims and rebates, so the majority verdict stands.

No verbatim quote is available for this clause.

Model votes: Kimi K3 missing, Grok 4.6 missing, GPT 5.6 Sol partial, Claude Opus 5 missing, Gemini 3.1 Pro missing

Three of five models read this as missing; Claude Opus 5 and Gemini 3.1 Pro called it bad because page 5 gives the vendor the sole right to audit participating pharmacies and to recover audit discrepancies from them. That clause is real and it does mean pharmacies face recoupment after adjudication, but it governs billing errors under the vendor's network agreements rather than authorizing steering or volume based pay. Pages 4 to 7 describe independent participating pharmacies and the vendor's own mail and specialty pharmacies without any anti steering promise, any willing pharmacy clause, any bar on volume conditioned reimbursement or any statement that claims are final when adjudicated. The subject the model clause covers is not addressed, so the majority verdict stands.

No verbatim quote is available for this clause.

Model votes: Kimi K3 missing, Grok 4.6 missing, GPT 5.6 Sol missing, Claude Opus 5 bad, Gemini 3.1 Pro bad

All five models reached this verdict. The consideration article on page 17 points to Exhibit B, the term article on pages 18 and 19 sets five years with a two year option and no repricing trigger, and page 13 allows performance standards to be adjusted once a year by mutual agreement, which is not a market check. Nothing promises the city pricing as good as comparable clients and there is no meet, credit or release remedy. The city's only lever during the term is termination on notice under page 29.

No verbatim quote is available for this clause.

Model votes: Kimi K3 missing, Grok 4.6 missing, GPT 5.6 Sol missing, Claude Opus 5 missing, Gemini 3.1 Pro missing

Four of five models read this as partial; Gemini 3.1 Pro called it bad because the limits are heavy. The city does have real rights: an annual claims audit and an annual rebate audit on pages 14 and 15, a post termination audit on page 30, annual SOC 1 reports and audited financial statements on page 17, and claims data through an online portal on page 7. Every right is then narrowed: one on site audit per plan year, an auditor who must be mutually acceptable and sign the vendor's confidentiality form, no access to pharmacy or manufacturer agreements, a cap of ten manufacturer contracts in the rebate audit whose terms may not be revealed even to the city, vendor comment on the draft report, and a sample of three hundred claims in the standard timeline on page 16. That is weaker than the model, which gives the plan its own auditor and full data on a schedule, but the rights exist, so the majority verdict stands.

Model votes: Kimi K3 partial, Grok 4.6 partial, GPT 5.6 Sol partial, Claude Opus 5 partial, Gemini 3.1 Pro bad

Three of five models read this as partial; GPT 5.6 Sol and Gemini 3.1 Pro called it bad because the vendor may sell deidentified plan data. Section 8.1 on page 19 gives the city ownership of information produced solely and exclusively for it, and section 22.5 on page 29 says all files are city property and must be delivered at termination, which is a genuine ownership clause. The same section 8.1 then reserves the vendor's trade secret rights inside that information and lets the vendor use, adapt and sell deidentified claims and eligibility data and share it with data integration firms, and page 29 lets the vendor scrub its confidential information from files sent to a successor. Ownership is present but the reuse ban the model contract requires is absent, so the majority verdict of partial stands.

Model votes: Kimi K3 partial, Grok 4.6 partial, GPT 5.6 Sol bad, Claude Opus 5 partial, Gemini 3.1 Pro bad

Two of five models read this as partial, GPT 5.6 Sol called it bad, and Gemini 3.1 Pro and Kimi K3 voted unclear because the guarantee schedule is in Exhibit A, which is not in the posted document. The body text on pages 13 and 14 is readable and settles the direction on its own, so under rule 3 the unclear votes are overruled by the substantive verdicts and the median of partial, partial and bad is partial. Page 13 says only city claims are used to measure claims operations, that penalties are settled once a year at the year end reconciliation, that the amounts at risk are the city's exclusive financial remedies, and that the total at risk is $175,000; page 14 takes any amount due off the vendor's invoice within sixty days. Guarantees exist and are plan specific, but they are annual, capped and exclusive rather than quarterly and dollar for dollar, so this is weaker than the model. Confidence is held down because the metrics themselves cannot be read.

Model votes: Kimi K3 unclear, Grok 4.6 partial, GPT 5.6 Sol bad, Claude Opus 5 partial, Gemini 3.1 Pro unclear

Four of five models read this as partial; Gemini 3.1 Pro called it missing because there is no liquidated damages clause and no two strikes exit. Section 3.4 on page 14 does make the penalty self executing: any amount at risk comes off the vendor's invoice within sixty days without the city proving damages, and page 27 requires misappropriated funds and system overpayments to be repaid within thirty days. The trigger is the vendor's own report of its failure, the amounts are capped and exclusive under page 13, late reports carry no automatic remedy, and termination for cause on page 29 needs a single thirty day cure with no second strike or regulatory integrity exit. Automatic remedies exist in a limited form, so the majority verdict stands.

Model votes: Kimi K3 partial, Grok 4.6 partial, GPT 5.6 Sol partial, Claude Opus 5 partial, Gemini 3.1 Pro missing

Three of five models read this as partial; GPT 5.6 Sol and Kimi K3 called it good because the amendment clause itself matches the model. Section 13.1 on page 20 and section 31.1 on page 34 do require any amendment to be written, later dated and signed by both parties. The economics can still move without a signed amendment: the MAC definition on page 2 lets the vendor amend the price list from time to time, page 8 lets the vendor add, remove or retier drugs on thirty days notice, page 9 lets it change the specialty formulary with quarterly notice, and page 1 lets it switch the pricing source after notice. The model clause bars economic changes by notice or updated list, so the majority verdict of partial stands.

Model votes: Kimi K3 good, Grok 4.6 partial, GPT 5.6 Sol good, Claude Opus 5 partial, Gemini 3.1 Pro partial

Four of five models read this as partial; Gemini 3.1 Pro called it good because the convenience exit is real. Section 22.2 on page 29 lets the city cancel without cause on ninety to one hundred twenty days notice, and page 19 lets it stop if funds are not appropriated, which is a genuine termination right. The same section makes the city repay a pro rata share of credits disbursed in the year of termination and of the implementation credit spread over the initial term, page 9 lets the vendor delay rebate remittance after termination, page 29 shifts record transfer costs to the city unless three years of the term are complete, and there is no regulatory integrity trigger. Those exit costs are what the model contract published by Mark Cuban and collaborators forbids, so the majority verdict stands.

Model votes: Kimi K3 partial, Grok 4.6 partial, GPT 5.6 Sol partial, Claude Opus 5 partial, Gemini 3.1 Pro good

Three of five models read this as partial; Gemini 3.1 Pro called it missing because no rate ceiling or acquisition cost rule appears, and Grok 4.6 called it bad because affiliated pharmacies keep purchase discounts. Section 1.13 on page 3 defines specialty by disease type and handling but leans on what the vendor uses, so the vendor draws the line. The plan runs on the vendor's Advance Control Specialty Formulary on page 9, which the vendor may change with quarterly notice, specialty is the one category exempt from the once a year removal limit on page 8, and specialty prescriptions run through the vendor's own specialty pharmacy under pages 4 and 5. No drug level ceiling and no acquisition cost pricing appears in the body, and any specialty rates would sit in the withheld Exhibit B. The subject is addressed and weaker than the model, so the majority verdict stands.

Model votes: Kimi K3 partial, Grok 4.6 bad, GPT 5.6 Sol partial, Claude Opus 5 partial, Gemini 3.1 Pro missing

All five models reached this verdict. Section 33.1 on page 34 makes costs and pricing data confidential information and has the vendor assert that parts of the contract itself are confidential. The Public Information Act clause on page 36 does not free the city: it must notify the vendor of any request, give it time to claim exemptions, not oppose the vendor's effort before the Texas Attorney General or a court, and release only the minimum ordered. Page 15 goes further and bars the rebate auditor from revealing manufacturer contract terms to any third party, including the city. The city may share data with its own consultants and auditors under page 35, but only under separate confidentiality agreements, so pricing, rebate and audit material is treated as proprietary in a way that blocks public disclosure.

Model votes: Kimi K3 bad, Grok 4.6 bad, GPT 5.6 Sol bad, Claude Opus 5 bad, Gemini 3.1 Pro bad