Boring dice testing

[Anderas]

What is the spec you are looking for with these dice? You are currently testing against a perfect random process, but how good is "good enough"?
I ask this because the more samples you collect, the greater the precision to which you are able to show that ANY die is not perfect. If you roll a million times, you could show that a die which is great for all practical purposes is outside of 5 sigma of a perfect die. In short, perfect should not be your objective.

I don't require perfection. If i invent monsters, if i make equipment, if i make a quest, i work under the assumption that in a game of Heroquest, you will not see differences smaller than 10%. That's because you do not roll often enough to "feel" the difference. That's different for games like 40k, but we're at the Heroquest forum here.
For example, if i make new equipment; making the next weapon a little bit more than 10% stronger than the one before, is for me kind of a prerequisite before i even think about producing and printing a new card.

To avoid hitting the 10% border accidentally with a die just because I don't want to test eternally, I would wish him to undercut that "feeling" border by half - so up to 5% away from the perfect die would be acceptable for me.
I know now, that those normal mass-produced dice are likely to cross this border more often than not.
You could avoid the entire problem if everybody round the table uses one common set of dice, so that every time you roll, you choose randomly among the dice set and it's failures. Only, that mitigation doesn't work at my home and in my experience because people take a "lucky" die and keep it for the rest of the evening. "That's my die now!" At least my daughter does. And who am i to intervene there? That would greatly overstretch the competences that myself being the daddy brings with it.

How bad are these dice really?

Bad. I see 3x more failures than expected. They are bad. Not a little bit, but really really bad. But i see light: The failure seems to be connected to the color. So sorting out and throwing away the failing sets should be easy.

Also, you have omitted the constraint in one of your assertions. It's not terribly important, but it does change the expected value for the number of results outside of a sigma value. Once you have one side which registers an extreme value (because of randomness or because of defect), the liklihood that the other sides will be wrong is not random. So there is not a 7.9% chance that a perfect die shows outside of 2.2 sigma, it is even lower.

I took only the most extreme % and calculated: How probable is it that at least one side shows extremes like that? So that's 100% - ((100% - extreme side) pow 6).

Now, to reduce the dice rolling work, i would like to calculate "how probable is it that this die shows more (or less) than the mean 4 out of 5 times?"
Worst case, i have to go with 50% 4 out of 5 times, best case i can somehow incorporate the Z-value/sigma that i have there.

[whitebeard]

What you want is a 95% confidence interval that any side is off by 10%.

What you have is a 95% confidence interval that the die is not absolutely perfect.

They are very different things.

[Anderas]

I didn't use confidence intervals at all as i have the theoretical basis available instead of a sample.

Well, i will check out what you say. Thanks for the feedback!

[benvoliothefirst]

THIS IS FASCINATING!!!

Thanks for all the hard work, Anderas!

[cornixt]

So now that you have this data, what are you going to do with it?

[Anderas]

The plan has two parts.
The first part is called the "realistic" part, the other one the "unrealistic" part.

The realistic part:
First, play around with it until i have sufficient material for my study's lesson write-up (done)
Second, play around with it until i am convinced that i found a method that's able to test dice without so much work.
Third, throw away the white, blue, clear and red dice; keep only the turqouise. I could do that right now, but i want to keep the opportunity for more data on bad dice.

The unrealistic part:
Wrap my Zombicide dice tower in some transparent material.
Drill a hole and fix it to a motor.
Use my Rasperry or my Laptop or any part from my box full of microcontrollers to turn it 360 degree and stop each time when it is upright.
Mount a PC Webcam or a mobile phone so that it has a good view on the dice tower.
Use my newly achieved python coding abilities to read the pips on the dice after each turn of 360 degree and write it to a csv file.
Drop the plan because it costs too much time.




About the second part of my first plan:
I guess i have to update my "operational definition" of a good die, and the calculation of my test results.

I want only dice that are showing more than 15.8% and less than 17.5% of each single number. That's 1/6 plus or minus 5% of 1/6.
I want to identify the dice that are off that threshold with 90% accuracy. That means, if i throw away a die, i have a 10% chance of throwing away a good die.
Now my head is steaming to find out exactly what can i do with my numbers to calculate this from my data.

Before, i was calculating the probability of a result being possible if the die was theoretically perfect, so my method currently is not the right one.

[whitebeard]

I want only dice that are showing more than 15.8% and less than 17.5% of each single number. That's 1/6 plus or minus 5% of 1/6.
I want to identify the dice that are off that threshold with 90% accuracy. That means, if i throw away a die, i have a 10% chance of throwing away a good die.
Now my head is steaming to find out exactly what can i do with my numbers to calculate this from my data.

Before, i was calculating the probability of a result being possible if the die was theoretically perfect, so my method currently is not the right one.

Glad to see you coming around.

You had said that you would accept a die if it was off by 10%... so if the mean was supposed to be 10, you would accept a side where the mean was actually 9 or 11, but not any further away. This would be 10% error in either direction.

I am very curious to find out "how bad" these dice actually are. And once you can calculate one result, you should be able to move the thresholds and make claims like.

With an 80% CI, 20 of 40 dice are outside of a 10% probability threshold.
With a 95% CI, 5 of 40 dice are outside of a 10% probability threshold.
With a 95% CI, 18 of 40 dice are outside of a 20% probability threshold.

Don't throw them away just yet! But if the results say you need to toss one red one, do know which one it is? Did you lable them?

Good luck!

[Anderas]

Yes, i put them in a box with a label, just in case. I mean, what's the use of such a work if you don't keep the results?
Maybe i sell them labelled instead of throwing them away. "Buy a die that prefers the six, unobtrusively" ... Well, but that would be immoral, wouldn't it be?




Do you think that i can use those subsamples with the central limit theorem to prove that the mean is off? Because, what i did see today is that the general mean is far more off than just one. So i don't really know if i can show a plusminus 5% or plusminus 10% border, to be honest.

I rolled 150 times to have a mean of exactly 25 at each number. That's nice and round as a number. Results were more, like,

1	28	25,3%
2	34	2,3%
3	25	50,0%
4	23	32,9%
5	13	0,39%
6	27	32,9%

The probabilities are the answer to the question "how likely is the result if this die was perfect?"
If you translate into confidence intervals, you can say it's 99.6% sure the die is not perfect.
If i just substract 5% from the positive extremes and add 5% to the negative extremes it doesn't really get better. Instead i am 99.1% confident that he's broken.
Is a die a "he", "she" or "it"? I think of them as persons.

So this die prefers the 2 and rolls less 5.

[cornixt]

Have you done a float test to see which sides are dominant?

[whitebeard]

What you need to do is compute the standard deviation for a side which is off by +10% and a side which is off by -10%. If you rolled three times you need the probabilities of 0, 1, 2, and 3, occurences. Then the standard deviation is readily computed. You rolled 150 times, so you will need to appeal to some clever solution... probably in a textbook and/or coded in excel already. I don't work with probabilities much, so my vocabulary is off. You already have the standard deviation for an unbiased die, however you computed that, just recompute for these boundarys.

If the result lies outside X standard deviations north of the higher probability mean, or X standard deviations south of the low... then you are certain it is out of spec (and not just barely in spec) to whatever confidence the end of the standard deviation curve implies.