W E I B AY E S T E S T I N G : W H AT I S T H E I M PA C T I F A S S U M E D B E TA I S I N CO R R E C T ? 1 David Nicholls, RIAC (Quanterion Solutions Incorporated) Paul Lein, RIAC (Quanterion Solutions Incorporated) Weibayes is a one-parameter Weibull analysis technique developed by Dr. Robert Abernethy and other engineers at Pratt & Whitney Aircraft in the 1970’s to “solve problems when traditional Weibull analysis has large uncertainties or cannot be used because there are no failures.”[1] A basic assumption governing the accuracy of Weibayes analysis and its associated test regimens, as stated by Abernethy, is that the value of the Weibull shape parameter, b, is known or can be reasonably estimated. Knowledge of b can be derived from historical failure data, prior experience, or from engineering knowledge of the physics of the failure. Engineering knowledge of failure physics and consistent use of accurate, representative b values is best supported by an historical library of Weibull beta plots based on actual corporate and product experience. Defining the Problem In the absence of “good” engineering knowledge of b, inaccuracies in the assumption of its value can have a significant impact on (1) the Weibayes test regimen used to define a reliability requirement, and (2) the interpretation of Weibayes test results used to demonstrate whether that reliability requirement has been met. Marketplace pressure towards decreased test times, fewer test samples, and zero- or sudden death-based test regimens are intuitively beneficial because they lower the total test cost and allow decisions to be made earlier. As one hypothetical example, assume that we are given a stated reliability requirement of 0.90 at 1000 hours (i.e., the B10 life). A Weibayes zero failure test plan is formulated using a sample size of 5 and an “assumed” value of 1.5 for b. The test time required to demonstrate that this requirement has been met is calculated to be 1533 hours per test sample. What are the corresponding implications and risk in the interpretation of the test results if the “true” value of b is something different? The purpose of this paper is to examine and quantify the risk of assuming an “incorrect” value of b that is higher than the “true” value when performing Weibayes zero-failure or sudden death testing, and the subsequent impact on the analysis and interpretation of the results. Reliability at a specified design life can be calculated based on Equations (1) and (2). From the standard Weibull equation: R( t d ) = e ⎛ t ⎞β −⎜ d ⎟ ⎝η ⎠ where, R(td) = Reliability at the design life, td td = Design life to be demonstrated h = Characteristic life at CDF = 63.2% b = Weibull shape parameter For Weibayes analysis, the characteristic life is expressed by Abernethy [2] as: 1 ⎡ N T β ⎤β η = ⎢∑ i ⎥ ⎣ i=1 r ⎦ where, Ti = Test time for each sample r = Number of failures N = Sample size h, b As defined above In order to determine an appropriate test time per sample, Equations (1) and (2) can be combined and rearranged to give: 1 β ⎡ ⎤β −t d ⎥ Ti = ⎢ ⎢⎣ N * ln( R( t d )) ⎥⎦ (3) Substituting the values from the Introduction yields a 1533-hour test requirement per sample for our Weibayes zero-failure test example to demonstrate that the design life of 1000 hours at 0.90 reliability has been achieved. There are other test options available to demonstrate this requirement, however. Sample sizes can be changed. As an alternative, since the “true” value of b is unknown, a different “assumed” value of b can be used. The question then becomes, what is the “best” test plan to use, given that the 1000-hour design life at 0.90 reliability is a firm requirement and there are cost, resource and schedule constraints to be considered. Using Equation (3), a range of possible test scenarios can be generated to provide visibility into options for sample sizes that include potentially “better” assumptions for b. Table 1 illustrates one example of potential zero-failure test scenarios, based on “assumed” values of 1.5 and 3.0 for the Weibull shape parameter. (1) 1 . This article is adapted, with permission, from the 2009 Proceedings of the Annual Reliability and Maintainability Symposium. © 2009 IEEE. THE JOURNAL OF THE RELIABILITY INFORMATION ANALYSIS CENTER (2) FIRST QUARTER - 2009 Design Life (td) = 1000 Hours Maximum Likelihood Estimate (MLE) Sample Size (N) Per-Item Test Time (Ti) to Demonstrate B10 Life (R = 0.90) b = 1.5 b = 3.0 1 4483 2117 2 2824 1680 3 2155 1468 4 1779 1334 5 1533 1238 10 966 983 15 737 859 Table 1: Weibayes Zero-Failure Test Scenarios Example For this hypothetical example, let’s say that organizational constraints dictate that we cannot afford to use more than 5 test samples, and there are significant schedule pressures to get this product to market. From the table, under the “b = 1.5” column, using fewer test samples requires additional test time per sample (unacceptable within our schedule constraints). Using more samples reduces test time per item, but this has already been excluded from consideration. One remaining option, since the “true” value of b is unknown anyway, is to assume a different value for b. This allows us to use the same number of samples, yet we gain some relief in the test time per sample (approximately 300 hours if we test concurrently, or 1500 hours if we test serially). In the competitive marketplace, every hour counts. How much impact can using this different value of b really have, anyway? The remainder of this paper will provide some insight into the answer. The Relationship Between Test Sample Size and the Beta Intersection Point If you review Table 1 for a given sample size and compare the required test times at b = 1.5 and b = 3.0, you will observe that, for sample sizes less than or equal to 5, the per-sample test times for the larger value of b are noticeably shorter than those for the smaller b. For sample sizes of “N” equal to 10 and 15, however, this relationship is reversed. What is the reason for this? Simply put, the intersection of the “assumed” and “true” beta plots represents a pivot point that can influence decisions about Weibayes test plans that may subsequently result in bad decisions and unacceptable, but unidentified, risk based on conclusions about the demonstrated reliability. The two parameter Weibull distribution [3] defines the Cumulative Density Function (CDF) as: F ( t ) = 1− e ⎛ t ⎞β −⎜ ⎟ ⎝η ⎠ (4) where, t = Time (in hours) h = Characteristic life at CDF = 63.2% b = Weibull shape parameter Mathematical manipulation of Equation (4) leads to Equation (5), which represents the straight-line solution to be plotted on Weibull graph paper [4]. ⎛ 1 ⎞ lnln⎜ ⎟ = β ln( t ) − β ln(η) ⎝ 1− F ( t ) ⎠ (5) ⎛ 1 ⎞ lnln⎜ ⎟ ⎝ 1− F ( t ) ⎠ ln( t ) = + ln(η) β The location of the intersection point of two Weibull solution lines with different betas (based on the same number of hours, t) requires that ln(t1) be set equal to ln(t2) and: ⎛ 1 ⎞ ⎛ ⎞ 1 lnln⎜ ⎟ = lnln⎜ ⎟ ⎝ 1− F ( t1 ) ⎠ ⎝1− F ( t 2 ) ⎠ Substituting ⎛ 1 ⎞ y = lnln⎜ ⎟ ⎝ 1− F ( t ) ⎠ (6) and equating the two beta equations (derived from Equation 5) yields: continued on next page ››› http://theRIAC.org 3 WEIBAYES TESTING: WHAT IS THE IMPACT IF ASSUMED BETA IS INCORRECT? continued from page 3 y= β1β 2 (ln(η2 ) − ln(η1)) β 2 − β1 (7) Table 2 provides a reference for the beta intersection point for various test sample sizes. For Weibayes zero-failure testing with equivalent test times, the number of failures, r, from Equation (2) is equal to 1 and the summation term becomes the sample size, N. Substituting the expression for h from Equation (2) into Equation (7) yields the two-beta equation: y= 1 ⎞ 1 ⎞⎤ ⎛ β1β 2 ⎡ ⎛ β2 β β1 β 2 − ln NT ln NT ⎢ ⎜( ) ⎟⎠ ⎜⎝( ) 1 ⎟⎠⎥ β 2 − β1 ⎣ ⎝ ⎦ (8) The intersection of the two beta lines can then be algebraically shown to occur at the By life percent value, where: By life % = e y *100% = 1 *100% N (9) This result is shown graphically in Figures 1 and 2 for a sample size of 5 and 10, respectively. Figure 2: Beta Intersection Point for Sample Size = 10 Sample Size (N) Beta Intersection CDF (By Life) 2 50.00 3 33.33 4 25.00 5 20.00 10 10.00 25 4.00 50 2.00 100 1.00 1000 0.10 Table 2: Beta Intersection Point as a Function of Sample Size Figure 1: Beta Intersection Point for Sample Size = 5 Each Weibull graph includes a line plotted at b = 1.5 and b = 3.0. For N = 5, the beta intersection point occurs at approximately CDF = 20% (or the B20 life), disregarding any inherent small sample bias that is associated with the Weibull Maximum Likelihood Estimate (MLE). For N = 10, the beta intersection point occurs at CDF = 10% (or the B10 life). Note that the beta intersection point is independent of both the design life, td and the test time, Ti (assuming that the test time on each of the “N” samples is equal). In other words, different values for the design life will shift the beta plot pair left or right along the x-axis, but the beta intersection point will not deviate from the By life value. THE JOURNAL OF THE RELIABILITY INFORMATION ANALYSIS CENTER The Relationship Between the Confidence Level and the Beta Intersection Point What happens to the beta intersection points when lower-sided confidence bounds are introduced into the analysis? Simply stated, with increasing confidence level the beta intersection point will shift vertically at design life, td. While it will be the focus of a future paper to determine the mathematical relationship that governs this shift, Figures 3 and 4, respectively, illustrate the concept. In this example, the design life is set at 1000 hours. The resulting CDF is 37% at the 80% lower confidence bound (LCB) and 49% at the 90% LCB (compared to the CDF of approximately 20% at the MLE based on the sample size of 5). FIRST QUARTER - 2009 ››In considering the use of confidence bounds, as the desired lower confidence bound increases, the CDF value of the beta intersection point also increases. Based upon these general relationships, we can now make some specific observations regarding the presence and quantification of risk associated with Weibayes testing if there is a difference between the “assumed” and “true” values of beta that govern the test length and the subsequent interpretation of the test results. Figure 3: Beta Intersection Point at 80% Lower CB Risk as it Relates to the Difference Between the Assumed and “True” Values of Beta As alluded to in the introduction and problem definition sections of this paper, there is risk associated with using an “assumed” value of b in establishing Weibayes test plans and interpreting Weibayes plots when the “true” value of b is something different. The risk discussed in this paper is limited to underestimating the required Weibayes test time to demonstrate a specified design life by assuming a value of b that is higher than the “true” value. First, a general statement: ››The higher the CDF level of the beta intersection point, the greater the risk of reaching an overly optimistic conclusion for a fixed By life below that intersection point. Figure 4: Beta Intersection Point at 90% Lower CB General Relationships of the Beta Intersection Point Upon examination, the following general comments can be made for a given design life: ››There is an intersection point between Weibayes plots of two values of b, below which the required test time to demonstrate that the design life has been met is shorter for the higher value of beta than the lower value (see Table 1 for n < 5). The opposite is true above the beta intersection point (see Table 1 for n = 10 and 15). ››The smaller the Weibayes test sample size, the higher the CDF value at which the beta intersection point occurs (see Figures 1 and 2 and Table 2). For the purposes of quantifying the risk, we will work with Figures 5 and 6. This hypothetical example is based on a B10 life requirement of 1000 hours. Due to organizational budget and schedule constraints, it was decided to proceed with a Weibayes zero-failure test based on a sample size of three. The organization does not have a Weibull library of beta values for its products, or adequate knowledge of the predominant physical failure modes of the design, so it needs to rely on engineering judgment to establish an “appropriate” value of beta for the test (the “true” value of beta is unknown). Best engineering judgment concluded that, at the 1000-hour B10 design life, the product would exhibit signs of wearout, so it was decided that b = 3.0 would be appropriate (especially since it resulted in a shorter test time that supported management budget, resource and schedule constraints). Based on these criteria, the three samples were tested for 1468 hours each (based on Table 1 for b = 3.0). continued on next page ››› http://theRIAC.org 5 WEIBAYES TESTING: WHAT IS THE IMPACT IF ASSUMED BETA IS INCORRECT? continued from page 5 failures for this example is to simply read the results directly from Figure 6. At T = 1000 hours, the “assumed” b = 3.0 line indicates a CDF of 10%, indicating that 10% of the population are expected to have failed by that time. The “true” b = 1.5 line at T = 1000 hours indicates that approximately 17% of the population will have failed. Figure 5: Quantifying Risk When “Assumed” Beta is Higher Than “True” Beta The test was run and no failures were experienced, so the organization was confident that the design life requirements had been met. Unfortunately, returns from the field during warranty did not support this conclusion. What might have gone wrong? If the organization had been in a position to apply the resources needed to collect and analyze their data and characterize the product’s dominant physical failure modes, they might have discovered that the “true” beta for their product was actually 1.5, meaning that the product wasn’t wearing out as fast as engineering judgment had assumed. Superimposing a Weibayes plot with a beta of 1.5 on Figure 5 provides some clarification of the impact of this assumption. From Table 1, the organization tested their product for the required time based on a sample of three and an “assumed” beta value of 3.0. If they had known that the “true” beta was actually 1.5, then they would (or should) have tested each of the three samples for 2155 hours (with zero failures) to support a conclusion that the design life requirement had been met. As a consequence, each sample was undertested by 687 hours (2155 hours – 1468 hours). The impact of insufficient testing on the conclusions drawn from the analysis can be read directly from the Weibayes plot in Figure 5. Instead of demonstrating a B10 life of 1000 hours, the testing actually demonstrated a B10 life of only 685 hours, which is 68.5% of the requirement. The analysis can easily be extended to determine the increase in the expected number of failures over any time period and population size of interest. Additionally, the increase in associated repair and support costs resulting from the lack of knowledge of the “true” beta value can also be determined. One approach to estimating the increase in the expected number of THE JOURNAL OF THE RELIABILITY INFORMATION ANALYSIS CENTER Figure 6: Increase in Expected Number of Failures Mathematically, the exact values can be found from Equations (1) and (2). Using the results from the analysis above and substituting them into Equation (2) yields: 1 ⎡ 3 1468(1.5) ⎤1.5 η = ⎢∑ ⎥ = 3054 hours 1 ⎦ ⎣ i=1 (10) where, 3 = Sample size, N 1468 = Test time per sample, Ti 1.5 = “True” Weibull shape parameter, b Substituting the value of h into Equation (1) results in: R(1000) = e ⎛ 1000 ⎞1.5 −⎜ ⎟ ⎝ 3054 ⎠ = 0.829 CDF = 1− R(1000) = 0.171 or 17.1% where, (11) 1000 = Design life, td 3054 = Characteristic life, h 1.5 = “True” Weibull shape parameter, b 0.829 = Reliability at the design life, td 0.171 = CDF at the design life, td = (1-R(td)) Suppose that there are 10,000 products (defined as “P”) in the field and it costs $5,000 to repair each returned item. The FIRST QUARTER - 2009 increase in the expected number of returns at 1000 hours, based on a “true” beta of 1.5, will be: ([CDF(btrue)*P] – [CDF(bassumed)*P]) = (1710) – (1000) = 710 additional returns (12) At $5,000 per repair, these additional returns will result in approximately $3.55 million in unanticipated cost to the organization. MS Excel® Spreadsheet Quanterion Solutions Incorporated (QSI), in its role in the operation of the Reliability Information Analysis Center (RIAC), has developed a MS Excel® spreadsheet based on the concepts presented in this paper. The spreadsheet supports: ››Calculation of a Weibayes zero or sudden death failure test plan as a function of (1) sample size, (2) required design life, (3) the value of “assumed” beta, and (4) the potential value of “true” beta. The output is the required test time to demonstrate that a design life requirement has been met under the stated conditions. ››Calculation of the error in the estimated design life based on the value of an “assumed” beta and the potential “true” beta. This error is translated into the “actual” design life based on the “true” beta in comparison to the “expected” design life based on the “assumed” beta. ››Graphical representation of the error and resulting design life for a range of “true” beta values in comparison to the single “assumed” value of beta ››Calculation of the increase (or decrease) in the expected number of failures over a user-defined time period based on the difference between the “assumed” and “true” value of beta. ››Simple calculation of the cost impact associated with the increase (or decrease) in the expected number of failures over the user-defined time period The spreadsheet is available for download from http://www. theriac.org/informationresources/demosanddownloads.swn the location of the design life CDF requirement in relation to the beta intersection point, and (3) the relative difference between the values of the “assumed” and “true” beta. This paper addresses only the risk associated with using an “assumed” beta value that is higher than the “true” beta value, as this represents the most damaging technical and financial risk scenario for the organization. As such, the statement can be made that the greater the difference between the values of the “assumed” and “true” values of beta (with the “assumed” beta being higher), the greater the organization’s risk through undertesting a product to demonstrate that a required design life has been met, and being overly optimistic in the interpretation of the demonstrated reliability. General recommendations that can be made through this investigation are that: ››Each organization should strive to thoroughly understand and differentiate the physical modes and mechanisms associated with the root failure causes of its products in order to identify their “true” beta values ››Each organization should establish its own library of Weibull plots and beta values that are representative of its specific products and applications, and are based on the physical modes and mechanisms associated with the root failure causes of its products ››Where cost and schedule permit, an organization should use the exact number of test samples that correspond with the By design life that is to be demonstrated, as that value determines the intersection point between the “assumed” and “true” value of beta. For example, to demonstrate that a B10 life requirement is met, use a sample size of 10, as the intersection of the “assumed” and “true” beta plots will be at the required design life (i.e., “no” risk if the “assumed” and “true” values of beta are different). References 1. Abernethy, R.B., “The New Weibull Handbook – Fifth Edition”, Dr. Robert B. Abernethy, June 2007, pg. 6-1 We have shown that there is a mathematically supported relationship between the intersection point for an “assumed” and “true” value of beta that can be used to assess and quantify risk when performing Weibayes zero failure or sudden death testing and analysis. 2. Abernethy, R.B., “The New Weibull Handbook – Fifth Edition”, Dr. Robert B. Abernethy, June 2007, pg. 6-2 3. Weibull, W., “A statistical distribution function of wide applicability”, J. Appl. Mech.-Trans. ASME, September 1951, 18(3), pg. 293-297 The level of risk is a direct function of (1) the vertical location of the beta intersection point on the Weibayes plot (mathematically proven to be a function of the number of samples tested), (2) 4. Nicholls, D. (Editor/Co-Author), “System Reliability Toolkit”, Reliability Information Analysis Center/Data and Analysis Center for Software, December 2005, pg. 526 Conclusions and Recommendations http://theRIAC.org 7

© Copyright 2020