Simulation and Six Sigma How to Complete a Six Sigma Project with Little of No Data (Or “Why Simulation Can Help Solve the Unsolvable”) Larry Goldman Decisioneering – Crystal Ball May 7, 2007 1 www.crystalball.com Simulation and Six Sigma Today’s Agenda The Problem: Little or No Data Simulation Basics & the Fit Within Six Sigma? Project 1: Loan Process Project 2: Inventory Optimization Project 3: Simulation with DOE Conclusion 2 www.crystalball.com Simulation and Six Sigma Why Do Bad Things Happen to Good Projects? • Staffing changes • Lack of strategic focus • No buy in from Process Owner • Not enough dedicated resources • Project takes too long & loses momentum • Lack of support from the top • …(what else?)… • Little or no data available 3 www.crystalball.com Simulation and Six Sigma The Less-Than-Ideal Project Time is not on your side. Takes way too long to: – Collect statistically viable project data – Implement process or design changes – Measure the effect of Improve solutions – Can only run limited DoEs – Find a solution in a competitive market Or your data… – Is too costly to obtain – Can only be estimated – Just doesn’t exist prior to project 4 www.crystalball.com Simulation and Six Sigma The Less-Than-Ideal Project The project X’s and Y’s are complicated: – Past data was not collected or poor measuring system – Many likely X’s (possible root causes in system) – Many non-normal distributions (skewed variation) – Physical models (generally designs) are impractical – Non linear equation is difficult to predict – Forecast relies on uncertain demand – Poor understanding of fluctuation of input values – Process becomes out of control despite “optimization” of inputs 5 www.crystalball.com Simulation and Six Sigma Healthcare Example: Healthcare software provider ran simulation to reduce cycle time on new software installation and implementation process by 50%. PROBLEM: Misys Healthcare Systems produces software used by physicians and hospitals. Installation and implementation of the Misys hospital enterprise software took 18 to 36 months, far too long. The team needed to design a new, faster process. SOLUTION: Given the long cycle time for this process, a simulation was the only viable option to determine the anticipated cycle time. The team simulated the high-level components of the new design to determine the capability of the new process and identify those elements in the new process that were having the greatest impact on cycle time. RESULTS: The project will allow for reduced cycle time and accelerated revenue recognition (in half previous time). The solution validated the intuition of the LSS team that the 50% cycle time reduction goal was realistic. The solution also provided a level of confidence to senior management that the team met their goals. 6 www.crystalball.com Simulation and Six Sigma Reliability Example: Simulation assesses Customer Reliability CTQ, finds the delinquent component in a complex system, and saves over 400 man-hours of calculations THE SITUATION Perform analyses on engines to determine reliability characteristics. The objective is to establish reliability predictions, evaluate how the variability of individual part reliability affects the system-level reliability CTQ, and make recommendations to Project Management Team (PMT). THE SOLUTION Build a model that accounts for the different distributions of the parts, including hundreds of assumptions and forecasts. Simulate over 400,000 trials to assess variability in failure rates and identify delinquent component. THE RESULTS - Save over 400 man-hours of calculations by automating analysis w/simulation - Determine that Customer reliability CTQ can be met 93.7% of the time - Identify the delinquent component that contributes the largest effect on system mean (24.3% ) and variability (97.3%) and recommend further analysis of this component to determine whether sub-components are at fault 7 www.crystalball.com Simulation and Six Sigma Process Control Example: At Motorola, a design process was brought back into control by simulating data to determine the hidden critical factor. PROBLEM: Field emission from carbon nanotubes (CNT) for display purposes was optimized using Design of Experiments (DOE). The brightness was improved by three orders of magnitude but the achieved gains could not be sustained in the “Control” phase of a DMAIC project, and the process reverted to poor performance. SOLUTION: It took an intense effort of circa two months to recover the process. Monte Carlo simulations were used to provide an excellent fit to all the measured emission data over the course of eight months in both range and shape. RESULTS: The simulations indicated the cause of the process drift. A hidden factor that was too time- and labor-intensive to measure in real-time was responsible and uncovered. With the aid of the simulations, the process could have been recovered within days instead of months. 8 www.crystalball.com Simulation and Six Sigma Types of Simulation • Monte Carlo (Stochastic) Simulation: Random sampling experiment used to generate multiple scenarios. Each trial is a complete event. • Discrete Event Simulation: Time-based analysis. The operation of a system is represented as a chronological sequence of events. Each event occurs at an instant in time and marks a change of state in the system. • CAD-based Simulation: Tools that animate CAD designs to simulate motion. • Instructional Simulation: 3-D simulators used for testing and training. 9 www.crystalball.com Simulation and Six Sigma Why Do You Need a Model? • Models are an attempt to capture behavior and performance of business processes and products. • Simulation is the application of models to predict future outcomes with known and uncertain inputs. SIMULATION MODELS 1 F = m∗a 2 3 LO HI Control Inputs Noise Variables Y = f (x) Y = f (x) Outcome Predictions 10 www.crystalball.com Simulation and Six Sigma Where Do Models Come From? 16d 0T τ= π d0 4 − di 4 ( Models come in many different forms • Regression equations derived from historical data (e.g., transactional processes) • Design of Experiments (DOE) response equations from measured observations • Mathematical relationships based on established physical principles (e.g., Shear stress in torsion tube) • General knowledge of business system or product (e.g., expert opinion) Data comes from the same sources. 11 www.crystalball.com ) Simulation and Six Sigma What Is Monte Carlo Simulation? • Definition: A system, or sampling method, that uses random numbers to measure the effects of variation or uncertainty. • The inputs: Probability distributions that represent variable or uncertain X’s (assumptions). Inputs can be defined by: – Existing process or design data (best) – Limited data (e.g., DoE, process with long cycle time) – Expert opinion (little to no data!) • The outputs: Any response / Y / formula / effect (visual forecasts) • The tool: Desktop simulation programs 12 www.crystalball.com Simulation and Six Sigma Inputs: Probability Distributions y Simulation requires probabilistic inputs. y Distributions use ranges of values and assign a likelihood of occurrence for values (e.g., a normal distribution could represent variation of the part dimensions). Probability Range Parameters 13 www.crystalball.com Simulation and Six Sigma Outputs: Charts and Tables Number of simulation trials Parts within the spec limits are shown in blue, parts outside spec limits are shown red Certainty (probability) that the forecast lies between LSL and USL Upper Spec Limit (USL) 14 Quality Metrics such as Cpk, ZST, p(N/C), etc.... www.crystalball.com Simulation and Six Sigma Sensitivity Analysis: A Critical Tool • Examine which few critical factors (X’s) in your analysis cause the predominance of variation in the response variable of interest (Y) – like a Pareto Chart • Operates during the simulation, calculating the relationships between all X’s and Y’s • Acts as communication tool to help team understand what’s driving defects and where to focus (or not to focus) your improvement efforts 15 www.crystalball.com Simulation and Six Sigma Stochastic Optimization Simulation can help you to understand and reduce variation but does not by itself offer the best solution. An optimization model answers the question "What's best?" rather than "What happened?" (statistics), "What if?" (simulation) or "What will happen?" (forecasting). The combination of simulation and optimization lets you make the best (optimal) decisions while accounting for the variability or uncertainty inherent within a process. 16 www.crystalball.com Simulation and Six Sigma Where Do Professionals Apply Monte Carlo Simulation in Six Sigma? 1. Product and Process Design (Little to No data) – Robust design is required. – Tolerance analysis is performed. – Process is relatively simple. – Project success and/or process risk are uncertain. 2. Project Management – Project has financial or schedule uncertainty. – Project has cost controls. – Project is high risk. 3. Systems Model Exists – You have a quantifiable process or spreadsheet. – Mathematical relationship exists. – Physical models are impractical. 17 www.crystalball.com Simulation and Six Sigma Typical Applications for Monte Carlo Simulation and Optimization • Process Optimization • Sales Forecasting • Market Sizing & Penetration • Cost Estimating • Tolerance Design/Analysis • Critical Parameter • Material Selection • Risk Analysis • Design for Variability • Reliability Studies • Product/Service Launch • Total Life Cycle Cost • Resource Allocation • Inventory Optimization • Value Stream Analysis • Queuing Analysis Identification • Project Selection • Strategic Analysis 18 www.crystalball.com Simulation and Six Sigma Project 1: Loan Process Improvement 19 www.crystalball.com Simulation and Six Sigma Problem Statement Identify Value Value Stream • A financial organization wishes to use Lean Six Sigma techniques on increasing the efficiency and decreasing the variation of their Loan Process. • Customer: Loan Applicants Improve Flow • Note: This could really be any simple Customer Pull process or sub-process / cell Process Perf. 20 www.crystalball.com Simulation and Six Sigma Project Overview by Phase Identify Value - Define Problem Value Stream - Improve Flow Create high-level process map Refine process map to include variation (distributions) Measure or estimate process step variation Monte Carlo Simulation to predict variation Determine variation drivers w/ Sensitivity Analysis - Address drivers and reiterate simulation to improve flow Customer Pull Process Perf. 21 www.crystalball.com Simulation and Six Sigma Step 1: High-Level Process Map Identify Value Value Stream Improve Flow 1 4 2 Customer Inquiry 5 Loan Underwriting Loan Application Loan Closing 3 6 Document Verification Loan Disburse • Delays and Rework in Loan Process do not add value to customers. Customer Pull Process Perf. • Use Process Map and Value Stream techniques to identify delays and rework (assuming all identified process execution steps are Value-Added). 22 www.crystalball.com Simulation and Six Sigma Refinement of High-Level Process Map Execution of Process Delay Rework Decision OUTPUT Measure INPUT Define Analyze • Unfortunately, high-level process maps generally Improve • Using Monte Carlo techniques, we can model the Control do not consider delay times or rework cycles at each process step (“Hidden Factory”). variation in execution & delay times, in addition to defects (reworks) occurring at each high-level process step! 23 www.crystalball.com Simulation and Six Sigma Refinement of High-Level Process Map • Six Steps • Four can be broken into Execution and Delay • Three rework loops • Upper Spec Limit = 96 hours 24 www.crystalball.com Simulation and Six Sigma Step 3: Measure or Estimate Process Step Variation Identify Value As part of the Value Stream Phase, an estimate or measurement of the process step times needs to be captured: Value Stream • Sampling: Samples of steps 1 and 6 indicate these steps vary lognormally and normally, respectively. • Expert opinion: No reliable measures of Steps 2 through 4 Improve Flow Customer Pull exist so expert opinion is utilized – Step 2 has a most likely, a minimum, and a maximum estimated process time – Step 3 has an 80% chance of being anywhere between 16 and 32 hours and a 20% chance of being anywhere between 32 and 48 hours – Step 4 can be anytime between 1 and 8 hours Process Perf. • Collection System: No data was measured for Step 5 so a measurement collection system was put in place for 100 processed loans. 25 www.crystalball.com Simulation and Six Sigma Building the Model - 1 For Execution inputs, define each step as the appropriate distribution ? 26 www.crystalball.com Simulation and Six Sigma Building the Model - 2 For Delay, define each step as an Exponential distribution For Delay, define each step as a Yes / No (Binomial with 1 trial) 27 www.crystalball.com Simulation and Six Sigma Building the Model - 3 • Now, just calculate Cycle Time (91 hours with delay and no rework) • Cycle Time = Execution steps + Delay steps + Rework when it occurs • Can also calculate VA Efficiency (is it always that high?) 28 www.crystalball.com Simulation and Six Sigma Step 4: What Does the Simulation tell us? After simulating 10,000 loans: • Mean loan process cycle time is 93 hours (vs. base case of 91 hours) • Standard deviation = 40 hours! • ~40% of loans (3,839/10,000) are over USL • Sigma level is a dismal 0.084 • As-is state has serious problems. What is driving the variation? 29 www.crystalball.com Simulation and Six Sigma Monte Carlo Simulation to Predict Variation Identify Value Value Stream VA Efficiency is reduced by including effect of added Cycle Time due to delay times and rework cycles (non-value-added steps) – VA Efficiency mean less than 100% (~ 35%) Improve Flow Customer Pull Process Perf. 30 www.crystalball.com Simulation and Six Sigma Step 5: Review Sensitivity Analysis Identify Value Value Stream Improve Flow Customer Pull Process Perf. • Run Sensitivity Analysis to determine major driver of variation. • Can anything be done to reduce Document Verification Delay times? – Assume average delay time can be reduced by 50% in Cell K33. – Run simulation. 31 www.crystalball.com Simulation and Six Sigma Step 6: Reiterate Monte Carlo Analysis Identify Value Value Stream • Run Monte Carlo again → less than 20% of process loans are out of specification → Sigma Level of ~ +0.8 • The Loan Process Cycle Time quality has been improved. Improve Flow Customer Pull Process Perf. 32 www.crystalball.com Simulation and Six Sigma Reiterate Monte Carlo Analysis Identify Value • By reducing the primary non-value-added Cycle Time variation (Verification Delay), the Value-Added Efficiency mean has also been increased (from ~ 35% to ~ 40%)! Value Stream Improve Flow Customer Pull Process Perf. 33 www.crystalball.com Simulation and Six Sigma Comparison of Results Stage Mean Cycle Time Mean VA Efficiency Standard Deviation Sigma Level Base Case 91 hours? 31.5%? ??? ??? As-Is Sim 93 hours ~35% 40 hours .08 To-Be Sim 75 hours ~40% 26 hours .83 Analysis is iterative and the model will be adjusted (improved) as the project continues… 34 www.crystalball.com Simulation and Six Sigma Project 2: Inventory Optimization 35 www.crystalball.com Simulation and Six Sigma Problem Statement • The two basic inventory decisions that managers face are: (1) how much additional inventory to order or produce, and (2) when to order or produce it. • Although it is possible to consider these two decisions separately, they are so closely related that a simultaneous solution is usually necessary. • Given variable (uncertain) demand over a 52-week period, you need to determine an optimal order quantity and reorder point that results in the lowest possible total annual costs. • Demand is estimated for each week, based on expert opinion or limited data. 36 www.crystalball.com Simulation and Six Sigma Project Overview by Phase Define - Review problem statement Measure - Create and validate system model Analyze - Characterize current process state with simulation - Determine variation drivers w/ Sensitivity Analysis - Address drivers and reiterate simulation Improve Control - Optimize process for cost and performance - Implement changes in ordering process - Moving forward, process owner compares results with simulation results, adjusts model as needed 37 www.crystalball.com Simulation and Six Sigma Step 1: Create Excel Model Define Measure Analyze • Determine amounts for inventory and ordering Improve • Create calculation for whether or not to place order Control • As-is state: $7,090 in Annual costs for order of 250 units • Calculate individual weekly costs and roll up to annual costs and reorder of 250 units 38 www.crystalball.com Simulation and Six Sigma Step 2: Define Key Assumptions All 52 weeks have same Poisson distribution for demand 39 www.crystalball.com Simulation and Six Sigma Step 3: Run the Simulation Define Measure Analyze Improve Control • After 10,000 trials, find that mean annual inventory costs is around $25,500. • The base case of $7,090 is far from realistic given the uncertainty of demand. 40 www.crystalball.com Simulation and Six Sigma Running Simulation with Optimization • Define objective: minimize mean of annual inventory costs • Define controllable variables: Order Quantity (200-400 units) and Reorder Point (200-400 units) Optimization (1 = 1000 trials) Order Quantity Reorder Point Minimized Cost (mean) 1 250 265 $18,474 2 345 320 $2,791 3 325 275 $7,705 41 www.crystalball.com Simulation and Six Sigma Running Simulation with Optimization Define Measure Analyze Improve Control • After 10 minutes, optimization has converged on Order Point of 330 and Reorder Point of 325. • This will minimize the Annual Costs to a mean of ~$2825. 42 www.crystalball.com Simulation and Six Sigma Running Simulation with Optimization Define Re-run simulation with new controls and see optimized inventory problem at 10,000 simulation trials. Measure Analyze Improve Control 43 www.crystalball.com Simulation and Six Sigma Project 2 Conclusions • Modeling demand of as-is state can show weaknesses of base case estimations for forecasts with uncertainty. • Stochastic optimization lets you run simulations while changing controlled variables for each consecutive simulation. • By adjusting controlled variables during optimization, you can determine settings that will optimize your output (e.g., minimize costs, maximize profit). • Final optimization solution results in reduced inventory waste and substantial cost savings. 44 www.crystalball.com Simulation and Six Sigma Project 3: Simulation with DOE 45 www.crystalball.com Simulation and Six Sigma Problem Statement Define Measure Analyze Improve Control • Situation: An Injection Mold Process has resulted in incomplete filling of the mold or different part lengths. A Six Sigma Project team has been assigned to reduce the variation not meeting length requirements. • Customer: Part Buyers • Approach: – Perform 23 Full Factorial DoE (5 replicates) to determine Response Surface model of Part Length – Use Crystal Ball Capability features to predict current quality metrics – Use OptQuest Optimization techniques to determine process settings that minimize process cost while meeting minimum quality targets. 46 www.crystalball.com Simulation and Six Sigma Project Overview by Phase Define - Review problem statement Measure - Measure current parameter capability Analyze - Perform Design of Experiments Characterize current process state with simulation Determine variation drivers w/ Sensitivity Analysis Address drivers and reiterate simulation Improve - Optimize design for cost and performance Control - Run capability study on proposed process settings to confirm quality 47 www.crystalball.com Simulation and Six Sigma Step 1: Measure Current Parameter Capability Define As part of the Measure Phase, the variation of the Control Parameters (Inputs, Factors) is characterized during Capability Studies Measure – Input Factors are Mold Temp, Cycle Time, and Hold Pressure – 30 samples of each are made during the studies and Factors are assumed to behave normally Analyze Each set of samples passes Normality Test Means and Standard Deviations are recorded Improve 25 20 18 16 20 14 15 12 15 10 10 Control 8 10 6 5 4 5 2 0 140 150 160 170 MoldTemp 180 190 0 48 80 90 100 CycleTime 110 120 0 120 124 128 132 HoldPres 136 140 www.crystalball.com Simulation and Six Sigma Step 2: Perform Design of Experiments Define • 23 Full Factorial DOE with 5 replicates is performed (40 runs) – RESPONSE: Part Length Measure Analyze Improve – FACTORS: LO HI Mold Temperature (x1) 100 200 Cycle Time (x2) 60 140 120 140 Hold Pressure (x3) • Response polynomial equation developed (R2adj = 92.5%) – 3 Main Effects – 1 Interaction Term Control Y = β0 + β1x1 + β2x2 + β3x3 + β23x2x3 49 www.crystalball.com Simulation and Six Sigma Step 3: Characterize Current Process State Define • Define the Inputs (Factors) as Normal Assumptions (Cells E5:E7) – Cell Reference Assumption Name from Column B Measure Analyze – Cell Reference Assumption Mean from Column F – Cell Reference Assumption StDev from Column G • Define the Response (Length in Cell E9) as a Forecast – Cell Reference the LSL from Cell F9 Improve – Cell Reference the USL from Cell G9 • Run Simulation Control 50 www.crystalball.com Simulation and Six Sigma Monte Carlo Simulation to Predict Variation Define Nominal Response of 64.59 mm close to target but 2% will fall out of the spec limits! → Sigma Level of ~ 2.0 Measure Analyze Improve Control 51 www.crystalball.com Simulation and Six Sigma Step 4: Review Sensitivity Analysis Define • Run Sensitivity Analysis to determine major driver of variation. Measure Analyze Improve Control • Can anything be done to reduce standard deviation of Mold Temperature? – Assume standard deviation can be reduced by 50% in Cell G5. – Run simulation. 52 www.crystalball.com Simulation and Six Sigma Step 5: Reiterate Monte Carlo Analysis Define Measure • Run Monte Carlo again → ~ 1% are out of specification → Sigma Level of ~ 2.5 • The Part Length quality has been improved – Can it be improved even more while minimizing cost to run the process? Analyze Improve Control 53 www.crystalball.com Simulation and Six Sigma Step 6: Optimize Design for Cost & Performance Define Measure How can the process settings be configured so that a minimum quality goal is reached while reducing the process cost per part? Analyze Improve Control 54 www.crystalball.com Simulation and Six Sigma Optimize Design for Cost & Performance Define Measure Analyze Improve Control • Must consider relationship between process parameters and cost. – Energy consumed by molding equipment is proportional to product of Cycle Time and Mold Temperature ($ ∞ Temp * Time) – Labor Cost to run molding equipment proportional to Cycle Time ($ ∞ Time) • Create Cost Response as a function of – Cycle Time – Mold Temperature $PROCESS = K1*Temp*Time + K2*Time • Define Process Cost Forecast (Cell E10) 55 www.crystalball.com Simulation and Six Sigma Process DoE Optimization Define • Characterize Current Quality Levels (Cpk & ZST) – Enable Capability Metrics in Run Preferences Measure Analyze Improve Control – In Define Forecast, use cell references for LSL & USL and auto-extract Capability Metrics • Assuming you can control the nominal process settings but not the variation, use Optimization to determine the settings that results in the best quality (maximum Zscore) • Process Parameters – Mold Temp → LO (100) to HI (200), Step = 10 – Cycle Time → LO (60) to HI (140), Step = 1 – Hold Pressure → Step = 2.5 www.crystalball.com 56 LO (120) to HI (140), Simulation and Six Sigma Helping You Optimize: Decision Variables Decision variables are Crystal Ball model elements for quantities over which you have control (e.g., percentage of dollars to allocate in a project, amount of product to produce, man-hours required for a project, unit cost for a given product, go/no-go decision). 57 www.crystalball.com Simulation and Six Sigma Define Decision Variables Define Measure Analyze Improve Control • Define Decision Variable Lower and Upper Bounds of all Factor means (Cells E5:E7) by cell referencing corresponding adjacent cells: Cell reference Name from Column B Cell reference Upper Bound from Column C (LO) Cell reference Lower Bound from Column E (HI) • Ensure the correct Discrete Step Size is used within each Decision Variable as listed below Decision Variables Lower Bound Upper Bound Discrete Step Size Mold Temp 100 200 10 Cycle Time 60 140 1 Hold Pressure 120 140 2 58 www.crystalball.com Simulation and Six Sigma OptQuest: A Blend of Approaches OptQuest excels at stochastic optimization because it: • Uses several optimization techniques (Scatter Search and Advanced Tabu Search) vs. relying on a single method or genetic algorithm, • Employs heuristics (problem solving techniques that use selfeducation to improve performance), • Has both short-term and long-term Adaptive Memory, • Can escape local optimal solutions to find global optimal solution, • Uses neural network technology that predicts performance after only running 10% of simulation and typically reduces number of required simulations by 50%, and • Features a wizard tool that makes setup easy. 59 www.crystalball.com Simulation and Six Sigma Optimize Design for Cost & 4σ Performance Define • Run OptQuest and Define Forecast Selections Optimization Goals: Measure Analyze – Primary is to Minimize Cost – Requirement is to Reduce Variation of Part Length to 4σ levels Zst required to have a lower bound of 4 Improve Control 60 www.crystalball.com Simulation and Six Sigma Optimize Design for Cost & 4σ Performance Define New Design results in a Process Cost of $1.16 per part and increase to 4σ quality! Measure Analyze Improve Control 61 www.crystalball.com Simulation and Six Sigma Comparison of Design Performance & Cost Define Measure Analyze Improve Control Where have we been, and where are we going? Iteration # Mold Temp Mean Mold Temp StDev Cycle Time Mean Cycle Time StDdev Hold Press Mean Hold Press StDev Sigma Level of Flow Rate Process Cost 1 160 10 100 10 130 5 1.94 $2.03 2 160 5 100 10 130 5 2.53 $2.03 3 150 5 61 10 140 5 4.01 $1.16 Six Sigma team proceeds to run Capability Study on proposed process settings to confirm quality during Control phase. 62 www.crystalball.com Simulation and Six Sigma Project 3 Conclusions Define Measure • Quality Levels will be increased by decreasing variation on driving input variables. – Monte Carlo analysis predicts quality levels. – Sensitivity analysis identified Mold Temperature as most influential design variable. Analyze • Knowledge of variation drivers allows one to Improve • Stochastic Optimization of input variable experiment with the process in the simulation world and determine improvements. (Factor) means will increase Part Length quality levels while minimizing Process Cost impact. Control 63 www.crystalball.com Simulation and Six Sigma Benefits of Simulation in Six Sigma Projects with Little or No Data • Provides a virtual recreation of the Process or Product needing improvement, even when data is estimated. • Can be used as a scoping tool early in DMAIC to guide project direction and project management issues. • Establishes current capability (as-is state) and tests potential improvements (to-be state). • Identifies defect-producing process steps driving unwanted variation (as well as CTQs). • Avoids extended wait for post-improve results and potential high cost of implementation. • Eliminates costly redesign-and-test loops and automates search for optimal solution. • Leads to Greater Customer Satisfaction 64 www.crystalball.com Simulation and Six Sigma Next Steps? • We will be here for the remainder of the event and can give demos and answer questions. • Trail versions (30 days) and informational materials are available with this event. Larry Goldman Decisioneering – Crystal Ball [email protected] 303-626-0129 • Visit the Crystal Ball Web site for free Web seminars, white papers, example models, and more. 65 www.crystalball.com

© Copyright 2019