Description
Beta test participants who reported SAT scores are listed by tracking number below. Scores obtained after SAT recentering in 1995 were converted to the pre-recentered scale using the table provided by ETS. The scatter diagram for the entire sample of 102 participants who reported SAT scores is shown below the table. Tracking numbers prefixed with "bmg" belong to students at a high school in southern California who were administered a paper version of the beta test and given one hour to complete it. Scatter diagrams for the "bmg"-only sample of 43 participants are also shown (SAT V+M, SAT V, and SAT M).
Problems with the Internet data
The data for tracking numbers which are not prefixed with "bmg" rely upon self-reported SAT scores from Internet participants. The obvious shortcoming with self-reported scores is that people can misremember their score or report only the highest one if they took the SAT multiple times. To try to minimize this problem, the test design team asked people to report all of their SAT scores and to resist the temptation to report only their highest scores. Where multiple SAT scores were reported, the first score was chosen, except if the participant was younger than 16 when the first score was obtained.
There is the possibility that Internet participants submitted answers multiple times. The test design team tried to minimize this problem by emailing tracking numbers to potential participants. Email addresses were checked for uniqueness, and only people with tracking numbers could take the test. Another disincentive to multiple submissions was the fact that solutions to the test were revealed and explained.
A standard time limit is difficult to enforce via the Internet. The test design team sought to minimize this problem by setting the time limit long enough to effectively eliminate it as a factor that could significantly affect test scores. The first small group of beta test participants served to determine what that time limit should be.
There are other, perhaps more significant, problems with this data. The ages of the non-"bmg" participants are not uniform. Although it is possible that the ages of the participants do not affect their test scores, it is more likely that they do.
Non-native English speakers were not identified. This is an obvious and serious shortcoming which the test design team should have identified prior to the beta testing, but didn't.
SAT scores are assumed to mean the same thing from year to year (after adjusting for recentering), but this assumption will introduce some error, since the SAT does in fact vary from year to year.
Problem common to both bmg and Internet data
The data does not represent a random sampling of the general population. The bmg data sample represents students from only one high school, and as such is probably more uniform than data that would be obtained from sampling many different high schools from various states. A more uniform sampling would tend to raise the correlation between the two tests.
The Internet data sample also deviates from a random sample. This sample is composed of people who volunteered to take the Beta test.
Interpreting the correlations
Even given the assumption that the Beta Test correlates well with the SAT, there are two interpretations of the data which do not necessarily obtain, and one should exercise caution when stating that:
1. The Beta Test correlates well with IQ.
2. The SAT correlates well with IQ.
SAT Adj. Age at
Tracking number V+M Score BetaTest Comments
bmg0000000000001 940 10.25 17 v = 520, m = 520 (recentered)
bmg0000000000002 920 11.25 17 v = 440, m = 550 (recentered)
bmg0000000000009 890 12.50 17 v = 440, m = 550 (recentered)
bmg0000000000010 1170 21.00 17 v = 550, m = 690 (recentered)
bmg0000000000016 1080 22.50 16 v = 600, m = 560 (recentered)
bmg0000000000021 1060 18.00 17 v = 530, m = 610 (recentered)
bmg0000000000022 950 17.00 17 v = 450, m = 590 (recentered)
bmg0000000000025 1160 21.50 17 v = 580, m = 650 (recentered)
bmg0000000000029 1230 20.25 17 v = 630, m = 660 (recentered)
bmg0000000000031 1080 28.00 16 v = 550, m = 610 (recentered)
bmg0000000000033 1020 24.25 17 v = 570, m = 550 (recentered)
bmg0000000000036 820 9.00 16 v = 470, m = 470 (recentered)
bmg0000000000037 940 15.75 17 v = 490, m = 550 (recentered)
bmg0000000000038 1080 22.25 17 v = 590, m = 580 (recentered)
bmg0000000000040 1070 14.50 17 v = 590, m = 570 (recentered)
bmg0000000000041 880 22.50 17 v = 480, m = 510 (recentered)
bmg0000000000046 1170 27.00 17 v = 650, m = 600 (recentered)
bmg0000000000047 870 22.00 17 v = 440, m = 530 (recentered)
bmg0000000000051 1200 20.50 17 v = 600, m = 670 (recentered)
bmg0000000000052 1160 28.25 17 v = 650, m = 590 (recentered)
bmg0000000000056 1170 28.00 17 v = 590, m = 650 (recentered)
bmg0000000000057 1270 31.75 17 v = 650, m = 680 (recentered)
bmg0000000000059 1110 28.00 16 v = 570, m = 620 (recentered)
bmg0000000000066 1330 27.50 17 v = 680, m = 710 (recentered)
bmg0000000000068 790 4.00 17 v = 520, m = 410 (recentered)
bmg0000000000070 1000 13.50 16 v = 500, m = 590 (recentered)
bmg0000000000073 830 21.00 17 v = 440, m = 500 (recentered)
bmg0000000000075 1180 32.75 17 v = 610, m = 640 (recentered)
bmg0000000000078 1050 23.25 17 v = 510, m = 620 (recentered)
bmg0000000000079 880 9.25 17 v = 520, m = 480 (recentered)
bmg0000000000080 990 18.50 17 v = 540, m = 550 (recentered)
bmg0000000000081 1160 22.75 17 v = 590, m = 650 (recentered)
bmg0000000000086 1240 24.75 17 v = 590, m = 730 (recentered)
bmg0000000000097 1100 19.50 17 v = 600, m = 590 (recentered)
bmg0000000000101 1060 14.75 17 v = 570, m = 580 (recentered)
bmg0000000000102 890 10.75 17 v = 460, m = 530 (recentered)
bmg0000000000104 820 8.00 17 v = 400, m = 520 (recentered)
bmg0000000000110 1120 26.75 17 v = 620, m = 580 (recentered)
bmg0000000000112 1360 28.00 17 v = 660, m = 790 (recentered)
bmg0000000000116 1140 27.75 17 v = 530, m = 680 (recentered)
bmg0000000000120 1290 24.00 16 v = 660, m = 690 (recentered)
bmg0000000000121 990 11.75 17 v = 580, m = 520 (recentered)
bmg0000000000122 830 6.75 18 v = 540, m = 420 (recentered)
2196185055653323 1490 31.50 28
2161095247511035 1330 28.25 28 Used 1st score, 2nd score = 1370
3299748835871771 1160 21.00 19 1230 (recentered)
3470532959516923 1310 28.75 25 Taken twice in 1990 -- only remember highest score from either subtest
3480222848892376 1450 31.25 28
3355295928847932 1580 35.50 24
1188548346401508 1490 18.00 41
1786708905906701 1340 30.25 20 1410 (probably recentered)
1919013902022125 1250 33.50 26
1419871823757401 1390 40.00 31
1238885170199335 1150 (not recentered; SAT thrown out -- Non-native English, flu during test)
1821056929212673 1210 22.75 22
1146322064275345 1480 40.00 23
1260265027420451 after '94 (SAT thrown out -- verbal ceiling score)
3457596632271494 1420 30.00 35
1463325734220871 1420 35.25 39
1283975410823901 1488 30.75 46
1839315102725285 1390 30.00 18 v = 680, m = 780 (recentered)
2348336775093164 1280 31.00 24 v = 700, m = 640 (probably recentered)
3036795701298548 1500 40.75 30
5699351346953903 1450 28.75 23
2237123167590993 1300 30.25 19 v = 660, m = 710 (recentered)
1600677816000525 1130 25.00 20 1200 (probably recentered)
2628260669866947 1430 30.00 38
2318615897266363 1290 23.50 30
5431052785134576 1320 39.00 41
2993950983913301 1302 25.25 49
8245826840610495 1230 27.50 24
2983884190521251 1340 40.00 19 1410 (recentered)
1828312420001889 1340 38.75 29
8240392440869529 1310 34.75 23 Used 1st score (2nd score = 1450)
8953081726108483 1470 41.00 18 v = 780, m = 770 (recentered)
1012787905525399 1460 39.00 34
1781603788600499 1250 24.50 20 1310 (recentered; chose score obtained at age 17)
1251637288101194 1410 34.25 17 v = 700, m = 790 (recentered)
3223760435918138 1080 30.00 21 1170 (recentered)
1943813546708032 1340 34.50 17 1390 (recentered)
4124644379538490 1125 24.50 28
1177338589847755 1170 23.50 34
1473585936674644 1485 13.75 44
3564642219199700 1440 37.75 34
9511410451738714 1420 31.25 25
3691476060673406 1030 27.25 21 Chose score obtained at age 16 rather than age 15
4596356667871533 1050 20.75 33
3320617089874484 1580 30.50 22
5118199584603128 1320 30.00 19 v = 780, m = 610 (recentered)
5170701183198764 1250 33.25 26
1040426405760655 1350 38.75 27
7943197696670206 1390 37.25 22
6132170002301016 700 11.25 26
2918353274421091 1410 41.00 28
6598668534742748 1370 28.75 35
6702385436511266 1310 22.50 28
5774595529833124 950 23.50 24
6651226012444344 1210 24.75 23
6220669990371125 1240 22.25 25
1663701974363843 1290 26.75 40
6476817535418661 950 17.00 30
6719686842220036 1250 23.00 29
6601534439402256 1330 40.75 24
2779481125022675 1343 35.50 45
r = 0.774
N = 102
