Roger D Metcalf DDS, JD

PO Box 137442

Fort Worth, TX 76136-1442

ph: +1-817-371-3312

roger

Calculate exact p-value for F-ratio and Chi Square

*How to actually calculate it--if you don't trust computers. *

*But you still have to use a computer.*

On a companion page on my website I demonstrated the calculations needed to find the exact p-value for a *t*-test by hand, if one is inclined to do something like that. And “by hand” I mean getting the function into a form that I could potentially evaluate with a calculator if I had to (but I'm going to ask Maple2020® to actually do the complicated math for me). I got interested in doing these computations in a statistics class I'm taking under Dr. Mark Payton in my Forensic Science PhD program at Oklahoma State.

Here, I work through the calculations to find the exact p-value for an F-ratio derived from an ANOVA. Again, I wanted to know *how to compute* the p-value *by hand, *and not simply how to go look it up in a table.

As with my other evaluation, to get started I first turned to the 29^{th} edition of the *CRC Standard Mathematical Tables and Formulae. Tables? In a "book?" *Yeah, sometimes we still look up things the old-fashioned way. Beginning on page 516 we find several pages of "percentage points" of F-ratios categorized by α-level, and then each individual table is arranged by the two degrees of freedom,

So, what do we do? Well, from our computer ANOVA output, we should have been given the F-ratio of interest (the F-ratio is actually trivial to calculate by hand, if you want to do that). So, let’s say we find we have F = 3.0. We also know our degrees of freedom, say for this example, *m* = 2 and *n* = 10. (I know, I know, your statistics software *already* gave you a p-value in the ANOVA, but I want to know *how* it was calculated.)

We go to the first F table on page 516 of the CRC book [below] and find that for degrees of freedom *m* = 2 and *n* =10, *at the *α* = 0.10 level*, the critical F ratio is 2.92. Write that down--and note the α level of 0.10.

Fig. 1. CRC table for critical F numbers at α = 0.10

(Note that this is doing things sort of "backwards" from the way we do it for a *t-*test. Here we* know* the degrees of freedom and a given α level, and we want to go find the critical F number. Using these tables the old-school way, we (1) go to the first table, the one calculated for α = 0.10, find our degrees of freedom of 2 and 10 [the organge circles above], see where they intersect [the green circle above], and write down the F value found there...for this example, it's 2.92, which is less than our own 3.0, so, (2) we keep going until we find an F that's bigger than ours at our degrees of freedom [below]. Eventually, (3) we find the two "critical" F-values that "trap" our own particular F-value, and note their associated α-levels. Remember, again, that's not *our* F-value from *our* ANOVA that's in the first and subsequent tables, those are the critical F-values found in the tables for our degrees of freedom and the various α levels. The "critical" F number is the number we will compare to our F number in order to make our decision about statistical significance.)

Ok, back to work--*our *F-ratio is 3.0, and we know that's *bigger than the critical F of 2.92* we just found, so we have to keep going, we’re not done yet. Look at the next table on page 517. Here, for our degrees of freedom *m* = 2 and *n* =10, and now α = 0.05, we find a critical F of 4.10, and this one is, in turn, *bigger* than our F number. So this is where we stop. Write that one down along with it's α level, too.

Again, what we want are the two critical F values that "trap" our 3.0 (with degrees of freedom 2 and 10), and that's the 2.92 and 4.10 numbers from the tables--our number falls between those two.

Fig. 2. CRC table for critical F numbers at α = 0.05

Further, if we really wanted to, we could look up all the critical F values for our particular degrees of freedom and construct a table for their respective α levels--these are all the pertinent α levels provided in my copy of the CRC handbook, other books may have additional tables at other α levels:

Fig. 3. Table of critical F values from the CRC handbook.

Now, since our F-ratio falls between the critical F numbers 2.92 and 4.10, that means the the p-value we want to know is thus somewhere between the respective α levels of 0.10 and 0.05--probably (interpolating “by eye”) closer to the 0.10 end of the scale. Not all that helpful in nailing down the *exact* p-value. (But those critical F values certainly *do* help us, of course--since our F of 3.0 is *larger* than the critical F of 2.92, we can say *our F number is significant* at the α = 0.10 level, and therefore we would *reject* the null hypothesis at that α; conversely, because our F of 3.0 is *smaller* than the critical F of 4.10, we say *our F number is* not* significant* at α = 0.05, and we would *fail to reject* the null hypothesis at that α [or at any α smaller than that]. *But I digress--my quest is how to get the exact p-value.*)

As before, I captured the function as printed in the CRC book using Maple Calculator (an app for iPhone) and uploaded that to my Maple cloud account. Maple Calculator uses the iPhone camera to capture an image of the printed function, does some pre-processing on the equation, and then uploads it to the cloud. * Very* nice.

Here’s the function as it appears in Maple2020 after importing it from the CRC handbook and sorting out the details to get it to work:

Fig. 4. F function from CRC handbook as entered in Maple2020.

Great. But, as before with calculating the p-values for the *t-*test, what do I do with those Gamma functions that are all over the place?

Well, as long as we're calculating Gamma for a positive whole number, that's easy-peasy.

So let's first look at the denominator—if *m* and *n*, the degrees of freedom, are *even* positive integers, then, ok, evaluating those two Gamma functions is straightforward. Fine, but now look at all the stuff going on up in the numerator—that’s just not going to wind up being a nice simple integer to work with except in a very few cases, if at all. I can’t Gamma that on my own. And if *m* or *n* in the demoninator is *odd*, I can't do those Gammas by hand, either.

So here’s how to calculate the exact p-value using integrals (the equivalent integral form of the Gamma function is found on pp. 350-51 of the CRC handbook I'm using):

Fig. 5. F function with Gamma functions replaced by their equivalent integrals.

Alright, alright, alright‼! I have a shot at evaluating that by hand, if I really wanted to--I could do the math--I could--LOL, probably--(ok, not with my old slide rule, but maybe with a calculator)--but, again, Maple2020 will kindly handle all the calculations for me instantly.

The main idea is, though, I have a *much* better grasp on what's going on in the analysis here than I would have had by just looking at the p-value spit out by the statistics software. And this also demonstrates one of the values of software such as Maple2020. While I claim I could do the math, Maple handles it much better and ** so much faster** and

Remember, our example has F = 3.0 and degrees of freedom *m* = 2 and *n *= 10, so I asked Maple to "evaluate at a point" and, upon entering those numbers…

Fig. 6. Eureka! The quest for the p-value is over! Beautiful.

Awesome. Notice that, LOL, Maple2020 converted my integrals right smack back into those dad-gummed Gamma functions…yeah, but that's ok, Maple knows how to deal with 'em!

If you want to see the actual numbers put into the integrals, here you go...

* * Fig. 7. The numbers from our example as substituted into the integrals and evaluated by Maple2020.

We want only the right tail, so remember to subtract that resulting number above in Figs. 6 & 7 from 1 to get the p-value that you actually want--and this is a one-sided test, so you don't have to worry about whether to double it or halve it like with some of the other tests:

**1 - 0.9046325684 = 0.0953674316.**

Awesome. So there you have it. You can easily capture the above images of the equations, if you wish, with Maple Calculator and an iPhone and shoot them right into Maple from the cloud—assuming you have Maple and a Maple account.

**©2021 Roger D Metcalf. All worldwide rights reserved.**

*The computations here were performed using Maple 2020®.*

*Maple is a trademark of Waterloo Maple, Inc.*

**References**

Maple2020. Maplesoft, a division of Waterloo Maple, Inc., Waterloo, Ontaro.

Percentage points, F-distribution. (1991). *CRC Standard mathematical tables and formulae, 29 ^{th} edition. *W.H. Beyer, ed. Boca Raton, FL:CRC Press, Inc.

**Now, you may ask, "why the interest in these statistical things?"**

* For one thing, I just really like doing statistics. In my PhD program I actually have a declared minor in Statistics--not many PhD programs have real "minors," but mine does.*

**And, not to be too coy, it may also have something to do with dental age assessment. Stay tuned..... **

I'm also tacking on computation of the exact p-value for the Chi Square distribution. It turned out to be almost kind of trivial after getting the calculations of p- for the *t*-distribution and F-distribution working, so here it is in brief.

This is what I wound up with after importing the function into Maple2020 with Maple Calculator as described above:

Fig. 8. Density function of p-value for Chi Square distribution.

..and, again, I know it's small, but you can click and zoom on the image. Here, *n* is degrees of freedom, and *x* is the Chi Square value. Maple had a bit of trouble with the form that appeared in the CRC handbook, because the book used the actual symbol for Chi Square and Maple2020 kept trying to interpret that as *x*^2. No worries--I just changed the symbol to plain *x* in the above and then everything worked fine. --> next column-->

** Maple is a trademark of Waterloo Maple, Inc.**

*The computations here were performed using Maple 2020®.*

**References same as above. **

So, here's what Maple said:

Fig. 9. More math. Remember, click on the image and zoom.

...uhh...mmm...no thanks, don't think I want to tackle that with my antique HP-67 calculator. I don't even know if I can do WhittakerM by hand--but pretty sure I can't :-)

Instead, again, switch to the integral form of Gamma, ask Maple to "evaluate at a point" (*n* = 5, *x* [the Chi Sq value] = 6.3), and...bingo! Note the integration is from 0 to the ChiSq value, so adjust your calculation accordingly when you want upper tail or lower tail, etc.

Fig. 10. Alright--now I'm done! Beautiful.

Ok, now that's something I could calculate by hand--maybe. With a calculator. Or maybe a computer. Hmmm, I wonder if I could get Maple to compute the probability that I could do the math by hand?

Anyway, it's pretty awesome.

© Copyright 2013, 2019 Roger D Metcalf DDS, JD. All worldwide rights reserved. No reproduction without permission. Neither the Tarrant County Medical Examiner's District, Tarrant County, the American Board of Forensic Odontolgy, the American Society of Forensic Odontology, the Royal College of Physicians, Oklahoma State University, nor any other organizaion mentioned here necessarily supports or endorses any information on this website. Any opinions, errors, or omissions are my responsibility, and mine alone. This site DOES NOT REPRESENT the official views of any of these--or any other-- organizations. Similarly, those other organizations may not fully represent my views, either.

Roger D Metcalf DDS, JD

PO Box 137442

Fort Worth, TX 76136-1442

ph: +1-817-371-3312

roger