Policy Selection Methodology

Technical appendix for the synthetic luxury-hotel comp decision prototype

Return to the executive brief · Download the executive PDF · Open the synthetic Decision Desk

Evidence boundary. This appendix evaluates policy rules on synthetic hotel operations and declared assumptions. Public Santa Monica Proper information supplies property-relevant gesture and guest-facing value context only. The result supports choosing a candidate for shadow validation; it does not establish actual policy effectiveness, savings, margins, or guest outcomes.

The question this appendix answers

The executive brief proposes a manager-facing Comp Decision Engine. This appendix explains how the prototype chooses which recovery rule is credible enough to test next:

Which candidate policy repeatedly protects the recovery obligation, preserves escalation and operational controls, and then has the lowest modeled cost under the tested assumptions?

The generated answer is Guardrailed recovery. It clears all declared guardrails in 99.6% of shared assumption-stress draws and is selected in 99.6% of those simulated worlds.

This is policy selection, not final predictive-model selection. The current candidates are explicit decision strategies applied to synthetic cases. A future real-data phase would separately compare manager judgment, transparent rules, interpretable statistical models, and nonlinear benchmarks on future-dated outcomes.

How selection works

1. Common cases x candidate policies
2. Calculate comparable metrics
3. Enforce hard guardrails
4. Stress uncertain assumptions
5. Rank qualified policies by stress-median cost
6. Select a shadow-validation candidate

How the candidate is selected. Guest recovery, escalation, data quality, and operational feasibility act as hard gates. Modeled cost ranks policies only after those protections are satisfied, preventing an inexpensive but inadequate policy from winning.

The comparison uses 430 common recovery cases and 5 policies, producing 2,150 matched case-policy evaluations. Each policy sees the same case facts and the same reference recovery-need tier. The decision strategy changes; the case mix does not.

Low-confidence reservation or CRM matches become data holds. Missing synthetic baseline actions remain unknown rather than being automatically labeled as under-recovery.

Candidate policies

Table 1: Candidate policies. Every rule receives the same recovery cases and recovery-need reference. The comparison changes the decision strategy rather than the underlying case mix.

Policy	Decision rule	Comparison role
Synthetic discretionary baseline	Reference-only replay of matched synthetic comp actions; missing comp records remain unknown rather than being labeled under-recovery.	Reference comparator only
Tiered standardization	Select the highest issue-fit tier-appropriate gesture without public context or cost optimization.	Eligible shadow-validation candidate
Guardrailed recovery	Require a robust recovery-fit margin first, then select the lowest midpoint-cost gesture; reserve direct refunds for cases without an adequate alternative.	Eligible shadow-validation candidate
Recovery first	Prioritize issue fit and guest-perceived value after operational constraints, using cost only as a tie-breaker.	Eligible shadow-validation candidate
Intelligent generosity	Use the current context-aware balance of recovery fit, estimated cost, guest value, demand, property fit, and operational pressure.	Eligible shadow-validation candidate

The synthetic discretionary baseline is useful for showing how inconsistent or incomplete historical behavior might compare with explicit rules. It is deliberately ineligible for selection because replaying synthetic history is not a proposed policy.

Metrics and hard guardrails

The design is non-compensatory: lower cost cannot offset inadequate recovery, missed escalation, infeasible gestures, or unresolved data quality. A policy advances only if all guardrails hold together in at least 80.0% of the assumption-stress draws.

Table 2: Metrics and advancement guardrails. All conditions must hold together. Strong cost performance cannot compensate for a failed recovery, escalation, data-quality, or feasibility requirement.

Measure	Operational definition	Advancement threshold
Safe recovery path	An adequate proposed gesture or an explicit manager-review path	At least 90.0%
High-risk under-recovery	High-risk case with an inadequate gesture and no manager review	No more than 5.0%
Operational infeasibility	Recommended gesture conflicts with timing or operating constraints	No more than 2.0%
Data-hold compliance	Low-confidence data matches are held for review rather than automated	100.0% compliant
Tier-5 review compliance	Every tier-5 case receives manager review	100.0% compliant
Joint stress reliability	Share of stress draws in which every guardrail passes together	At least 80.0% of draws

“Safe recovery path” is intentionally broader than gesture fit. It counts either an adequate gesture or a deliberate manager-review path. Strict gesture adequacy evaluates the proposed gesture alone. Reporting both prevents escalation from being mistaken for an automatically adequate comp.

Two uncertainty questions

One resampling method cannot answer every uncertainty question. The analysis therefore separates variability in the synthetic case mix from uncertainty in the assumptions used to score fit and cost.

Table 3: Two uncertainty methods. The paired bootstrap measures sensitivity to synthetic case mix; shared-world stress measures sensitivity to policy assumptions. Neither estimates real-world effectiveness without property outcome data.

Method	What changes	Question answered	Does not establish
10,000-draw paired case bootstrap	Recovery-case IDs are resampled once per draw and applied to every policy	Would the comparison change under another mix of the same synthetic cases?	Sampling uncertainty for Proper Hotels
5,000-draw shared-world assumption stress	Recovery weights, gesture fit, occupancy pressure, and cost assumptions vary together for every policy	Would policies still clear guardrails and rank similarly under plausible assumption changes?	A calibrated probability of real business success

The paired design matters: each bootstrap draw applies the same sampled case IDs to every policy. The shared-world design follows the same principle: each stress draw applies one coherent set of uncertain conditions to every policy. Policy differences therefore are not artifacts of evaluating competitors in different simulated worlds.

Synthetic post-stay scores are excluded. Their generator contains no comp-treatment effect, so using them to claim recovery effectiveness would manufacture outcome evidence.

Why Guardrailed recovery advances

Figure 1: Why Guardrailed Recovery advances. Lines show the P05-P95 modeled-cost range and points show the median across 5,000 shared assumption-stress draws. The baseline and Tiered Standardization do not reliably clear the declared guardrails. Among the surviving policies, Guardrailed Recovery has the lowest stress-median cost. These are synthetic comparisons under assumed costs, not Proper Hotels savings estimates.

Table 4: Final decision trace. Guardrails determine eligibility; stress-median modeled cost ranks eligible policies. Direct-refund exposure and manager-review burden apply only within the configured 1% cost tie range.

Policy	Evidence	Decision
Synthetic discretionary baseline	Pass 0.0%; P50 $19,109	Reference only
Tiered standardization	Pass 0.0%; P50 $25,914	Fails reliability gate
Guardrailed recovery	Pass 99.6%; P50 $30,467	Advance: lowest qualifying P50 cost
Recovery first	Pass 100.0%; P50 $51,114	Qualified; higher P50 cost
Intelligent generosity	Pass 100.0%; P50 $40,114	Qualified; higher P50 cost

The reference baseline is not eligible for selection. Tiered Standardization does not advance because it fails the joint reliability gate. Recovery First and Intelligent Generosity qualify but have higher stress-median costs. When the full selection rule is rerun within each stress draw, Guardrailed recovery is selected 99.6% of the time, Intelligent Generosity 0.4%, and Recovery First 0.0%.

The selected policy’s fixed-assumption midpoint is $29,104. The ranking quantity is instead the $30,467 median across shared stress draws, with a P05-P95 range of $27,342-$33,944. This distinction matters: the point estimate describes one declared assumption set; the stress median summarizes the policy across many coherent assumption sets.

Among policies clearing the 80.0% reliability gate, the lowest stress-median cost advances. Policies within 1.0% of that minimum are resolved by lower direct-room-refund exposure and then lower manager-review burden.

Guardrailed recovery is deliberately an adequacy-constrained cost optimizer. Its advantage is therefore a result of the declared guardrails, fit definitions, and cost assumptions. It is not independent evidence that the rule improves guest satisfaction or profitability.

What real data must establish

Shadow mode should first test whether the required fields can be joined reliably, whether recommended gestures are actually available, whether cost ranges match property accounting, and why managers accept or override the guidance. A four-week, minimum-50-case period is a workflow and instrumentation test, not proof of impact.

With credible operational data, compare four approaches on the same future-dated cases:

1. manager judgment as the operating baseline;
2. explicit recovery rules;
3. an interpretable statistical model with calibrated outcome estimates;
4. a nonlinear benchmark that must retain the same recovery and escalation safeguards.

Use temporal holdouts, calibration checks, paired policy comparisons, segment diagnostics with small-cell suppression, and actual marginal cost. Historical comps alone do not identify causal gesture effects because managers selected different gestures for different situations. Shadow observation and, when justified, a powered controlled test remain necessary before attributing guest outcomes to the recommendation.

Reproducibility and review

The appendix is generated from:

• comparison configuration: config/policy_scenarios.v1.json;
• matched evaluations: data/marts/policy_case_comparison.csv;
• policy summary: data/marts/policy_decision_summary.csv;
• uncertainty summary: data/marts/policy_uncertainty_summary.csv;
• evaluation engine: scripts/evaluate_policy_strategies.py.

Current comparison version: comp-policy-comparison-v1.0.0. Bootstrap seed: 20260721. Assumption-stress seed: 20260722.

make compare-policies sensitivity technical-appendix
make test validate public-audit

The implementation tests matched case-policy grain, policy-order-invariant shared draws, generated rather than hardcoded selection, uncertainty bounds, guardrail compliance, data contracts, and public-release safety.

Research anchors

• Hart, Heskett, and Sasser, The Profitable Art of Service Recovery.
• de Matos, Henrique, and Rossi, Service Recovery Paradox: A Meta-Analysis.
• Tax, Brown, and Chandrashekaran, Customer Evaluations of Service Complaint Experiences.
• Vargas-Calderon et al., Review-Based Quality-of-Service Framework.

These sources motivate disciplined recovery, fairness, timing, and review signals. They do not supply property-specific policy weights, costs, or outcomes.