I never guess. It is a shocking habit—destructive to the logical faculty. – Sir Arthur Conan Doyle, The Sign of Four
The Schapiro Group’s newest sex trafficking manifesto revolves around their highly dubious claim that it is possible to scientifically guess the ages of women in photographs with a degree of certainty that allows those guesses to be equivalent to fact:
The key to the technique described in the marble example [see yesterday’s column] comes from the phrase “if we knew from previous experience.” The problem is, there is no scientifically reliable previous experience on which to base the probability that a girl selling sex who looks quite young is, indeed, under 18 years. Therefore, we conducted a separate study to serve as this previous experience. Basically, the study involved asking a random sample of 100 adults to guess the ages of a variety of females in photographs. Some of these pictures were of females whose ages were known (teenagers to young adults), and some were not…the pictures of unknown girls came from erotic services postings on the Atlanta Craigslist web site…subjects were posed provocatively (e.g., a picture of a female licking her lips). Pictures of females of unknown ages were selected because the subject appeared “young.” In selecting the pictures, multiple reviewers agreed that there was at least some chance that each of the females of unknown ages in the pictures was actually under 18. This is how we operationalize “young” throughout the study. Study participants viewed each of these pictures and estimated the age of each pictured female. Importantly, study participants rated the average age of females from Craigslist (whose ages we did not know) the same as the average age of pictured females whose ages we did know. Study participants were balanced by race and gender, though the results indicated conclusively that participant demographics did not have an impact on age estimations, nor did the demographics of the pictured females have an effect.
…study participants tend to overestimate the ages of provocatively posed females…across all ratings of known-age females, participants tended to assume the females were 2.5 years older than they actually were. When a girl under 18 poses provocatively, participants tended to overestimate her age by 7-8 years, whereas when the subject was closer to age 22 or 23, the age estimate was much more accurate than the average overestimate of 2.5 years. In fact, women age 24 and over tend to be estimated as younger than they actually are when posed provocatively. This effect, which is represented by a curvilinear mathematical equation, allows us to speak definitively about the probability that a female of a given estimated age is actually under age 18. In fact, the study showed that any given “young” looking girl who is selling sex has a 38% likelihood of being under age 18. Put another way, for every 100 “young” looking girls selling sex, 38 are under 18.
Reread that if you need to; the truth is cleverly hidden, but there. Assuming that everyone could agree on what is “provocative” (which men and women don’t, but we’ll leave it there anyhow), the only ages of which the authors could be certain were the ones whose ages were known, none of whom were prostitutes! This experiment might have been somewhat valid if the ages of ALL the pictured women were known, but since the experimenters improperly introduced an “x” factor into what should have been a controlled experiment there is absolutely NO way to know which percentage of the girls were actually under 18. This is such a glaringly obvious mistake that I can’t believe the authors were unaware of it; what seems more likely is that they initially conducted a proper study which produced results too low to satisfy them (like New Zealand’s 3.54%, perhaps) and so were forced to redesign the study with an unspecified percentage of photographs of unknowable age (“some” is not a valid mathematical expression of percentage) in order to get the results they wanted. Simply put, there is no way for the authors to know whether the girls of unknown age (who, since they came from Craigslist escort ads, were presumably automatically considered “provocatively” posed whether they were or not) were 7-8 years younger than they appeared, 2.5 years younger or actually older; the 38% figure is therefore completely invalid even if 100 cherry-picked experimental subjects were a large enough sample to derive such conclusions (which they aren’t).
The paper then goes into a long obfuscation about escort services (designed, no doubt, to convince the reader that the authors know what they’re talking about) which as I discussed in my previous column on these scammers ignores the fact that the vast majority of escorts tend to revise their ages down. The section contains such portentous sentences as “Escort service operators have told our callers they have 17 year-old escorts specifically” and “we also know that many of these phone numbers go to just a handful of call centers.” Since the age of consent in Texas is 17 and many escort services have multiple phone numbers, these sentences actually have no semantic value but are included to make the services seem “shady”. The use of the term “call center” makes it sound as though a third party was answering the phone, which is entirely incorrect; multiple phone numbers go to one business, not multiple businesses to one external “answering service”. But even if they did, what of it? Many doctors may use the same answering service; does that make them criminals? The whole thing degenerates into a silly song and dance about “CSEC victims per service” which in the end translates (again) into “we guessed”.
The next section starts out with a statement which is either unbelievably ignorant or an egregious lie: “As of November 2010, the tracking data do not include any content from Craigslist, as it closed the ‘adult services’ section of its website in the U.S. Recently the story was completely different. There were many websites, but only one main source for paid sex services ads in states across the U.S.: Craigslist.” Yes, this paper is actually making the astonishingly stupid and easily disproved claim that prior to this year, there were essentially no other online sources of escort ads worth noting. Backpage, Eros and all the various hooker boards did not, according to the Schapiro Group, exist.
This stunning idiocy is followed by the comparatively subtle “There were an estimated 52 CSEC victims advertised each day across all major websites…the data show that many of these girls do not stay long on these sites, a finding that is consistent with the notion that many girls are trafficked state-to-state.” Now, this is a sensible statement if one makes the unwarranted presumptions that all young prostitutes are involuntary, controlled by others, and “trafficked” from place to place; unfortunately for the Schapiro Group, none of those presumptions are true. The reason many of the ads disappear quickly is very simple: Many young girls decide to try escorting, place an ad, discover in a call or two that they don’t like it, and never renew the ad. Every escort service owner has had to deal with young girls who quit after their first call or two; this is no different from any other entry-level job (telemarketing, for example, has a very high attrition rate). The only reason the authors’ assertions are not instantly perceived as ridiculous by their target audience is that they all buy into the underlying assumption that sex work is intrinsically different from all other work.
After a few more pages of mumbo-jumbo to justify still more guessing, the authors present their final numbers drawn from a hat, then claim that these self-generated numbers exceed the number of women who die by suicide, homicide, accidents, AIDS and childbirth combined. Yes, I realize that comparing prostitution with causes of death is like comparing apples to hamburgers, but obviously the Shapiro Group hopes its readers won’t catch the non sequitur. The rest of the paper consists of self-congratulatory statements about the “reliability” and “credibility” of their guesses (apparently they’ve never heard of that inconvenient thing called “peer review”) and advising readers on how to use the propaganda to convince legislators to divert money from programs dedicated to helping real victims by “provid[ing] you with a high degree of perceived credibility among various audiences.” I don’t think it would be inappropriate for me to apply the term “shameless” in this context.
My age has never been guessed correctly. Not once. The range of the majority of guesses is 19-24, and they only go as high as 24 after they’ve heard me speak (apparently my speech patterns are too old for nineteen).
Back-of-the-napkin math shows that, borrowing this “study’s” methodology, I am 21.8 years old. Someone needs to inform Vital Statistics so I can update my license at once.
Actual age – 32.
These errors are too egregious to be simple mistakes. The numbers has been made to whore itself out far worse than you or I have ever done. You’re completely correct here; the math just does not work, and anyone who can calculate simple percentages can see it.
It’s like I keep telling people in regards to the economy. Math doesn’t care what you’d really like to see, or that you’ve got great reasons or a good cause. Math has no emotion. It just is.
Good point about the methodology, Emily; I’m generally guessed as about 35, which by their “methodology” means I’m 33. Actually, I’m 44. The math simply doesn’t work.
Have you ever read “The Cold Equations” by Tom Godwin? It’s considered among the best science fiction short stories ever written, and is widely anthologized. If you haven’t read it, DO NOT look it up on Wikipedia because there’s an absolute spoiler. DO find the story and read it.
Em, not questioning your mathiness, just curious…
How did you get 21.8?
The closest I can get is 24-2.5 = 21.5.
Again, just curious.
Em and Maggie…
Taking the chart at face value (more on that later), I think I see where you’re getting your numbers, but we don’t really know what happens to the curve after age 25. Since your actual ages are (ahem) off the scale, the formula for the curve from SG would be really helpful.
Anyway,…
I’m sure they don’t give a damn about escorts over 25. And considering that the majority of escorts I’ve known are at least that age, and many looked quite young, is yet another flaw in the study.
Hee hee, “Your Mathiness” sounds like the way one might address the Mathemagician from Norton Juster’s The Phantom Tollbooth.
I used a baseline of 32 and a totally guesstimated figure which is close enough for government work of 200 people making guesses over the course of the last year. Accuracy would mean recording my exact age at the time of the guess (32 and three months, 32 and 8 months, etc) but, like I said…. back of the napkin.
Most guesses are between 19 and 21. I input a value for “most” that looked close enough to me, split all the values among the range as percentages, took the total mean inaccuracy and subtracted from actual age of 32.
I spent fifteen seconds on it, tops (labor of love, I tell ya), so feel free to correct the total.
Em,
heh? I still don’t get it, and I was curious, not correcting. I cannot correct that which I do not understand.
Maggie,
Good point. I also realized that by zooming into a small age range, they made the differences between the lines more pronounced. The cheesy is strong in this one.
I think you’re right. Shapiro seems to be creating a study that generates a predetermined result. It reminds me of a mail-order college degree.
I have to wonder if they could just as easily generate a study that proves the rarity of children in prostitution. My guess is that they could.
I’m positive they could. The example I used when talking about it to Grace last night was that they could devise a study to “prove” or “disprove” global warming, depending on who was paying them. It’s easy when your “study” isn’t peer reviewed and your audience already believes whatever it is you’re claiming to “prove”.
While in college I took a post graduate statistics class taught by a woman with a Ph. D. in statistics. She said that given a set of data about a particular topic she could both prove and disprove any assertion you could make about the topic the data was about (obviously data about the ages of prostitutes cannot be used in assertions about this year’s orange crop).
So you and Maggie (below) are both correct. With a data set that includes the ages of prostitutes, you can prove via statistical analysis BOTH that under aged prostitutes are rare, and common.
Hmm, when I replied to Dave’s post, it put my comment entry box below Dave’s comment, and above Maggie’s reply, so I assumed that was where my comment would go when I posted it. I was wrong. So my preceding post was intended as a reply to Dave, and my reference “… Maggie (below)…” should have been “… Maggie (above) …”.
So are you now sufficiently confused?
I’m not sure I really understand the methodology (I’d have to read the actual report, which I haven’t done), but what I’m guessing (and we know how bad that is, but still) from what you say is something like this:
The researchers had two groups of pictures: group 1, those of girls whose ages they knew, and group 2, those of girls whose age they didn’t know. All the photos were chosen because (regardless of actual age) the girls looked young. Now, it seems what they did was get their subjects to guesstimate the age of both group 1 and group 2 photos. On the basis of data from group 1 photos (whose ages they knew), they did an analysis of how often the subjects guessed right and how often they guessed wrong. Based on this, they came to the equation that allowed them to predict that a young girl in these photos had a 38% chance of actually being younger than 18. (I’m sure it wasn’t simple proportionality, since they talk about fitting an equation to the results, but the main point would be that this was done with group 1 data, i.e. girls whose age was known.)
Then they applied the same equation to group 2, the Craigslist girls whose age was not known. They assumed they could use their result — 38% — to group 2 without any adjustments.
Is that it? In your reading of the report, am I right? Is this what they did?
Because, if it is… it would seem that that is indeed more or less OK. It’s always problematic, of course, to apply results from one group to another if you can’t show that they belong to the same sampling population (if they come from different populations, there is no a priori reason to expect their statistical properties to be similar: they might belong to completely different distributions). But this isn’t in principle worse than applying the New Zealand result to American prostitues… unless one can show that there is some reason to doubt that the estimations of group 2 ages would be more prone to errors than the estimations of group 1 ages.
Again, do I follow this correctly? Or is there something I’m missing?
BS in mathematics. “Solve for x” only works when you know all other factors.
The two control groups are not known to be comparative, they are assumed to be. Group One held subjects who were known to be underage. The ages in Group Two weren’t known at all but it was taken as fact, with no evidence that fact existed, that some of them must be underage as well. Also, using pictures pulled off CL cannot be considered a reputable population sampling, as the controller can’t even determine that the person being studied (the picture) is in fact the same person being advertised. Fake pictures are everywhere on CL.
When Maggie compared New Zealand to America, she assumed that each country had both women and prostitutes and extrapolated the study of one country onto another. A bit of crudeness in the numbers will be expected, but her assumption was inherently valid. It’s the same methodology used by polling groups such as Pew Research Group and Gallup.
Finally, you can only aspire to mathematical accuracy with a large sampling data. I can prove anything with a cherry-picked sampling of 100, e.g., 92% of hobbyists are black men aged 32-39, 100% of politicians are convicted felons, 87% of policemen are wife-beaters. And I won’t have to fudge the math.
Classic case of “If it doesn’t have a tail, it’s not a monkey, it’s an ape” being used to prove a cow is a monkey.
That is indeed the weak point of the whole story, as I see it. How comparable are the groups supposed to be? One can of course be skeptical and demand more information about group 1 (the girls whose age was known); that might support the idea that the two groups can’t really be statistically compared. That’s what I would like to see, and that’s what I see no evidence of in their study.
The ages in Group Two weren’t known at all but it was taken as fact, with no evidence that fact existed, that some of them must be underage as well.
If I understand well, that’s not entirely correct: they apparently operationalized the notion “young” as “looking such that being underage “. I assume (am I wrong?) that both groups were chosen with this criteria in mind: the girls looked young enough that they might be underage. Or am I misreading something?
If I understand well, nothing was assumed about group 2 girls: some might be underage, some might not be, all might be underage, all might not be. Or am I again misreading something? Where is the assumption made?
Sure, and in the absence of better data relating to American women, that is indeed OK. But it seems to me that this is the same they did with group 1 and group 2, right? And in both cases the problem is: how sure can we be that both groups do pattern alike in terms of their statistical properties? How similar are really group 1 and group 2 — or Americans and New Zealanders?
The Schapiro group guys should have argued for that. I would expect a justification for why group 1 should provide an estimate usable for group 2.
That is indeed an extra problem. The “probability of dishonesty” should also be estimated and taken into account. (One possibility would be to actually contact a population of girls from CL ads to determine if they fit their photo, and then calculate the percentage of dishonest photos there.)
That is in principle true, but increasing the sample size beyond a certain limit ends up bringing less and less improvement to the results (because the accuracy actually increases with the square root of the sample size); after a certain number the expected improvement doesn’t justify the increase (depending, of course, on how much money your project has). The number I was told was 32: have at least 32 subjects, never less than that.
Now, the main problem here is “cherry-picked”. If the subjects were cherry-picked then of course they don’t constitute a valid sampling population, no matter how many there are. But were they cherry-picked? If there is evidence of cherry-picking, then of course the whole study is of course not trustworthy.
We are told nothing about the girls in the “known age” group;were they posed provocatively? Were they wives and daughters of Schapiro Group members? Were they professional models? Were they trying to look younger than they were, as most escorts try to? We also are not told what percentage of the pictures were known age compared to which percentage of escorts; if they looked at three pictures of known girls and 97 of Craigslist girls, or vice-versa, either case could be described as “some of each” but the results might be very, very different indeed. Finally, as Emily points out, we don’t know if the Craigslist girls were actually the genuine girls or just the pictures of models they downloaded from the internet; for all we know the escorts may have largely been in the 30s but were using pictures of teenage models. There are too many improper “x” factors to consider the “experiment” remotely scientific.
Indeed, Maggie, these are all pertinent questions about their methodology. In a serious paper on their results, they would need to have a logish section answering all these questions — and if they didn’t, the first peer review round would tell them to rewrite the section to address them. They’ve disclosed surprisingly little in the report. I wonder why; aren’t they proud of their “quantum leap” methodology? At least in my personal experience, whenever a researcher comes up with a new, cool experimental design s/he is proud of, s/he really wants to tell everybody all about it, so you can see how clever s/he is!…
Yes, but those are genuine researchers, which the Schapiro Group isn’t. They’re a marketing firm, whose assumptions are totally different. The problem is the same as the one which results when scientists are involved in legal cases. In science, one presents ALL information, even that which would tend to disprove his theory. But in an advocacy system (like law), one only presents the information which tends to support his position and is even allowed to HIDE information to the contrary unless the opposing side makes a formal legal demand that such hidden information be revealed. 🙁
In their own words, the goal of the study was to prove “the probability that a girl selling sex who looks quite young is, indeed, under 18 years.” The methodology assured that unless each and every person who took the poll was able to identify a known 17 year old’s age with infalliable accuracy, women of unknown age would be “proven” to be underage.
In detail, the assumptions are as follows-
– the presence of a known underage girl can prove the presence of a previously unknown underage girl
– that the girl in the picture is advertising sex for sale
The methodology and assumptions cannot prove the presence of an underage girl in the group whose ages are unknown. The study isn’t even meant to. If you look at the math, they’re studying the ability to guess a woman’s age from a picture. The only result they can give with any intellectual honesty is the probability of an incorrect guess and a fair stab at how far one might be off in either direction. Mathematically, that leaves us with an equation that looks like this, “62% probability of x(+2.7/-4.3) = <18, solve for x". And mathematically, that's a bogus request. It can't be done.
Sure, and in the absence of better data relating to American women, that is indeed OK. But it seems to me that this is the same they did with group 1 and group 2, right?
Not exactly. It’s be the same if Maggie had argued that the presence of prostitutes in New Zealand proved the presence of prostitutes in America without any prior knowledge of same, and then went on to show exactly how many there were and what age groups.
Agree completely. The only way I can see they could have made this study mathematically valuable is to have known the ages of all the women.
OK, I am officially fucking impressed. Remind me to consult you next time I have a feeling that something is wrong with the statistical methodology of one of these studies, but I can’t quite put my finger on what!
Yes, that’s the way I saw it as well. If one had a very large sample of pictures of models of known ages, posed specifically in the types of ways typically found in escort ads, of every age from say 12 to 48, then gathered together all those whom at least 10% of a large number of survey respondents had guessed the age as under 18 and compared the average guessed age with actual age, that would give you an average age-guess error which could be applied to a much smaller, unknown population of similarly-posed models whom at least 10% of the SAME respondents guessed as under 18.
Indeed, and even a cursory look at the report you linked to shows that they aren’t showing everything. That is not a publishable paper, by any stretch of the imagination; even if their results happen to be good, that report leaves a lot of unanswered questions. (I am myself making here some assumptions about “what they must have done” that are not confirmed by anything in the report — other than me being a nice guy and assuming they made reasonable research decisions; which as you point out is probably not the case for the Schapiro group).
Indeed, but there is reason to suppose that there are underage escorts, even if only at the New Zealand level of about 3.5%, right? You are right when you say that the study is really about determining the accuracy with which the subjects were capable of identifying underage women from photos; but as I see it it’s not necessary to admit perfect accuracy. If they were correct 38% of the time when guessing which group 1 girls were underage, the assumption is that they will be correct 38% of the time with group 2 as well. The problems I see here are:
(a) does group 2 really have the same age distribution as group 1, or does it differ significantly?
(b) the 38% accuracy was determined only with reference to group 1, i.e. it was tested only once; I would rather see a number of attempts — say, 10, 20 groups of “young” women, selected according to the same criteria — to check if their accuracy is consistently 38%. (It probably will be more like a random variable, with estimated value hopefully near 38%.)
That is indeed the case: the basic result here has to be the probability of a correct or incorrect guess of a “young woman”‘s age (selected by their criteria) as less than 18.
Now, after this, what they did was use the resulting probability to estimate how often they would be right about group 2 girls. They’re making the (possibly unwarranted) assumption that their efficiency would still be the same for group 2 girls, so 38% of the cases guessed as underage would be correct (even if nobody can really tell which ones).
Now in itself that’s not terribly bad — it’s what people do when they sample products from one factory and use the results to judge products of another factory. Ideally they would sample the other factory too, but if something, some real-world circumstance makes this sampling impossible, guessing with data from the first factory might be the only thing they can do. They should, however, be modest about it and point out all the problems inherent in applying to Factory 2 what they deduced from the output of Factory 1.
I don’t think so, actually; because it is already known that there are prostitutes, and underage prostitutes, both in America and in New Zealand. The point is really how many, not whether; so indeed it seems they’re doing the same thing. Maggie estimates how many underage prostitutes there are in America by using percentages calculated for another population (New Zealand); and this report tries to guess how many underage prostitutes there are by using guesses that were shown to be 38% correct for a certain group of women and then applying it to another group of women.
There are of course problems with doing that, and the Schapiro group (if they’re honest) should be upfront about it (any paper based on such data has to have one section describing in detail all the weaknesses of the methodology). Basically, the only reason to use such “go-to-group-1-then-apply-to-group-2” methods is that something makes it impossible to directly count group 2 — which would be any reasonable person’s obvious choice of methodology. (Is it really impossible for serious researchers to contact escorts and actually collect data on their ages? With anonymity procedures, nobody would know who exactly is what age.)
Indeed this would work, and it would avoid the guessing biases from photos that were not taken to be used in escort ads (presumably the group 1 girls). But then again, they talk about “provocative poses”; it may be that they did just that, i.e. that group 1 girls (whose ages were known) posed in provocative ways that resemble your usual escort ad. Again, I can’t tell because they actually say nothing about that. (In a more serious study, the photos in question should have been an appendix to the paper.)
I don’t think it would be hard at all, but it would provide a proper answer (which the Schapiro Group does not want). They want the largest possible answer which still can be pretended as being derived from objective data, just as the “researchers” in the Pepsi Challenge wanted to create an “experiment” which would result in the highest possible number of Pepsi preferences.
I had wondered if there would be legal problems, i.e. if researchers would be required by law to report to the police any underage escort they happened to find (even if by accident), and if this would lead to escorts lying in their answers. But if this is not so, if the escorts can be guaranteed that nobody is going to be turned in regardless of age, then it would indeed be feasible. (There is the problem of whether or not the escorts would tell their true age anyway — I’ll bet this is what the Schapiro group will say motivates their not doing it this way –, but there probably are ways to solve that.)
Someone should apply for money for this kind of research project.
It would never be given because the government doesn’t want to know the true answer, and independent foundations don’t want to get on the bad side of the government. Maybe Village Voice Media (owners of Backpage) might spring for it; it would give them ammo to use against their attackers.
I repeat: It’d be the same if Maggie had argued that the presence of prostitutes in New Zealand proved the presence of prostitutes in America without any prior knowledge of same.
The controller knew underage escorts existed somewhere in America. The controller did not know, and freely admits so, whether any of his second control group were underage. The controller then attempts to convince us he has mathematically proven 38% of his control group are underage, and therefore 38 of every 100 youthful-seeming women posting ads for negotiable affection on Craigslist are under the age of 18.
If you truly feel this is the same, then here is the equation: 62% probability of x(+2.7/-4.3) = <18, solve for x.
The other option is that the equation isn't possible. For the studies to be the same, the controller should have known the factors used rather than assume them. Maggie's math works. It's back-of-the-napkin, but it works. This group's math doesn't.
The only things I can think of on top of this is that, first, the appearance of the women must share traits seen among sex workers. Botox, Restalin, implants hither and yon, manicure, pedicure, etc. All designed to preserve the appearance of youth or exaggerate sexual qualities, and both of which will affect the accuracy of the guess.
The second is that they’ll then have to work their math backward. Math doesn’t work unless you can mirror it. So the smaller group upon which the equation is tested must then have exact ages determined, and the success rate of the equation on the individual and the group as a whole will be known.
No worries. I was always good at math. Pity I never cared for it as a career choice. About all I do with it these days is calculate 40% off at the store and laugh at mainstream economists.
emilyhemingway, but what you’re saying here:
is synonymous with saying that one cannot assume group 1 and group 2 will have the same distributions — i.e. it is synonymous with saying that the result obtained for Factory 1 cannot be used for Factory 2. That is what being skeptical about the two groups having the same distribution means.
(nit-picking: it’s not clear to me whether they mean “38% of all women” or “38% of the women who their subjects guessed were underage” — an important difference here that wasn’t clear from the text.)
The point is not that the equation involving a probability cannot be solved (about which you’re indeed right; it cannot). The point is that they cannot show that group 1 and group 2 represent the same kind of population, so guesses that worked for group 1 cannot be assumed to work for group 2.
Think about this: the implication that the fact their subjects were right 38% of the time when predicting which group 1 girls were underage means they will also be right 38% of the time about group 2 (escorts), no matter what differences there are between these groups, is tantamount to claiming that they would always be right 38% of the time for any group of young women selected according to their criteria for youth. Even, for instance, a group hand-picked to include only girls who are 18 or older.
Which shows why it is that they should convincingly argue that the two populations — group 1 and group 2 — are statistically similar. For all the reasons Maggie and you have pointed out, this begins to look more and more difficult to swallow.
emilyhemingway, Maggie’s math works only because we assume that New Zealand and America are similar with respect to the percentage of underage prostitutes. If we had different statistics from a different country — say, if we knew for sure that 38% for the prostitutes in Uganda are underage — we would have the same problem: which probability should we apply to the US? (We’d still pick New Zealand, on the assumption that New Zealanders and their society are closer to Americans in many respects in which Ugandans aren’t — but it is an assumption, to be later confirmed or disconfirmed with better data — i.e. a real census of prostitutes and their ages in America.)
Which is the same problem of deciding that a certain guessing strategy works for group 1, and then assuming it will work for group 2 — even though, as both of you point out, group 2 includes people who are doing their best to look younger than they actually are (which will probably confuse the guessers: they will get the wrong signals from whatever physical features their ‘unconscious guessing mechanism’ is guiding itself by).
I agree Maggie’s math works better — but because it seems to me that assuming New Zealdanders resemble Americans is more reasonable than assuming group 1 young girls resemble group 2 young girls (when we know group 2 is, among other things, trying to look younger).
emilyhemingway, a final thought.
Suppose they had obtained 100% success in guessing the ages of group 1 girls. Your equation would become:
100% probability of x(+2.7/-4.3) =< 18
which can be solved for x, because the probability is now a certainty and is thus canceled out:
x(+2.7/-4.3) =
x =
x =< – 28.666… .
Yet this would not solve the problem at all, because even 100% success rate in guessing which group 1 girls are underage cannot be applied to group 2 girls unless we know that they are statistically similar — which looks improbable since, as you and Maggie point out, the girls are trying to look younger (which will affect the physical cues on which the guessers base their guesses), etc.
In other words, even 100% success wouldn’t justify the transfer of information from group 1 to group 2 without a clear argumentation of why we should expect both groups to represent a similar challenge to the guessers. It seems less and less likely that they do, hence the problem.
Sigh… the html formatting brackets got confused with the ‘less than’ and ‘more than’ symbols by the parser here.
So it should be:
x(+2.7/-4.3) LESS THAN OR EQUAL TO 18, hence
x MORE THAN OR EQUAL TO – 28.666… .
All in all, it doesn’t seem that we disagree: the entire problem is extrapolating from group 1 results to conclusions about group 2. That’s what I read in your first post.
Even if they did the two separate groups, unless Group 1 was composed of women who will have greater success if they look younger than they really are (centerfolds, strippers, porn stars, even fashion models), it’s still ooky.
And now, for something utterly unrelated. Because of the unseasonably warm weather, I’m about to leave for a three-day weekend at the girlfriend’s. I’ve recently downloaded a manga with a funny title, and it’s led to some amusing mutterings as I get ready to go:
“Hhmmnn…. I need to check on the size of My Balls. If there isn’t room for My Balls in the flash drive, I’ll have to leave My Balls behind. Man, I’d really rather take My Balls with me.”
It’s more of a “real” age versus perceived age study, not a two group studies.
There are two groups of pics (known age and unknown age), but from what I’ve read, they are presented as one set of pics to participants in the study.
The two sets of pics do present big problems.
My Balls must be really good! 🙂
Hi. Writing from the GFs. Discovered something about My Balls: not as big as I thought. Plenty of room for My Balls on the ol’ flash drive, with room for Kodomo no Jikan left over.*
I’ve enjoyed My Balls so far. Once I’m done with My Balls, think I want to burn My Balls to disc. I sure don’t want to delete My Balls.
If you don’t mind supernatural and quasi-religious themes, tease-and-denial themes, a young man being gang-raped by hot women while trying desperately to avoid orgasm themes, etc., then you might also enjoy My Balls.
Um, I mean that in a totally not-gay way, of course.
* For the record: no, I am not “into loli,” but KnJ is such a sweet, often funny anime (which will never, ever, never ever ever never be on Cartoon Network!) that I wanted to read the manga, if I could find a fan-scan. It is not (big surprise!) available commercially in the US.
Now this is indeed a very astute observation. This is an effect in their population that they clearly did not take into account at all. This is a clear example of why such studies would benefit from actual contact with people who have first-hand experience in the business: they would hear about other potential influences on their results that they have to control for. They would avoid what is usually called “outsider mistakes.”
In an ideal world, any such project would have someone like you, Maggie, as a consultant: someone with first-hand experience in the area. Alas, that is clearly not the case here.
It’s worse than that. You’re presuming that the Schapiro Group is composed of social scientists who WANT to get real results but don’t know how. But that isn’t so; it’s composed of marketing experts who specialize in generating scientific-looking reports that prove whatever it is their paying clients want them to prove.
I’ll give you an example of this sort of marketing tactic. In the 1980s the Pepsi-Cola company wanted to come up with a way to ensure that their product would win in a taste test against the far better-selling Coca-Cola, so they hired a Schapiro Group-type marketing firm. This group discovered that when people drink an extremely cold beverage (just barely above the freezing point) the taste buds are numbed and the only basic taste the drinker can perceive is sweetness. Pepsi is sweeter than Coke, so if both are consumed at such cold temperatures the Pepsi has a more distinct taste while the Coke seems bland. The marketing company thus created the “Pepsi Challenge”, in which members of the public were given extremely cold samples of both Pepsi and Coke to drink, and of course most of the drinkers proclaimed Pepsi tasted better even if they normally (at typical drinking temperature of 5-10 degrees Celsius) preferred Coke. Thus the study “proved” that most people preferred Pepsi to Coke…even though it wasn’t true and still isn’t.
In only 11 states is the age of consent 18 and the only big state is California. In all others, it is lower – in Mississippi only 12 for non-virgins. This would seem to further undermine the concept of a girl “looking underage”. Under what age? The older you get the more girls look underage. For a geezer like me, any woman under 50 looks underage.
Exactly so. Years ago I had a business acquaintance with two daughters; I had met the 16-year-old and she looked 16, but then I was introduced to the other daughter, whom I took to be at least two years older. I said, “I thought B. was your oldest child,” and he replied “she is, this one is 12.” If a 26-year-old woman mistook a 12-year-old for an 18-year-old when meeting face to face, how could any man be expected to tell? The very idea is absurd. 🙁
Indeed. If telling a woman’s (or even a man’s) age were a simple procedure, why would people ever be carded at bars?
The only excuse they could have for doing this would be the claim that it would not be possible to actually gather real age data from real escorts. Maggie says this would indeed be possible; and it’s really a pity that nobody is doing that. It does look like the obvious thing to do.
Their reason for not doing it is that they know damned well 38% of young-looking escorts aren’t really underage, just like Pepsi-Cola company knows most Americans prefer coke. Marketers are not interested in truth, but in selling a product.
If the Schapiro group is like all marketing groups, then this certainly is the real reason why they don’t do it; but they can’t say so in polite company, so I’m assuming they will have to play the “but-we-couldn’t-get-real-data-by-just-asking-escorts” card. (I’m also guessing that, if someone did such a study, that’s the kind of criticism they’d get from the Schapiro group.)
[…] out the analysis of the analysis on Maggies page here. Now even my mathematically challenged brain knew something seemed off but was unable to figure it […]
There are some problems with the study that can be fixed. For example, they could more information about the known-age pics. How many pics are from known-age females? What is the age range of the known-age teenagers? What’s the formula for the curve?
Again, those problems can be fixed.
The big problem is getting provocative pics of known-age, underage females and their solution to this problem. I’m not sure if this could get past a review committee, but if it did, I wouldn’t want to be there when a photographer is asking an underage girl to lick her lips (this is from the SG report, not my twisted mind). Ideally, from a methodology perspective, KNOWING the age of ALL the females in ALL of the pics would work best. This might get past a review board and parents, but I wouldn’t bet on it.
The SG used reviewers instead. At first, I was thinking they could get a representative sample of average folks to review the pics. That could serve as a baseline or norm. But if they get a second representative sample of average folks for their study, there should only be minor, random overestimations or underestimations. The guesses should generally match. So, scratch that idea.
One way to do the study right, with pics from unknown age females, would be to get independent age identification experts to be the reviewers. This would cost money, but that’s how things work. Pics that have too wide a range in terms of age guesses (from t reviewers) would be thrown out. Accepted pics could be used in terms of the average person guesses (the study). This would bring it closer to being a valid study.
According to Hollywood, there’s software out there that can guess ages. I might use that instead of human raters, but the software might not exist (Hollywood).
Anyway, I agree that conducting a valid study probably wasn’t the goal.
Found another methodological flaw…
I did a quick check of their references (footnotes on page 4) first.
Their focus is on estimation of “provocative” pics. Why not get a baseline for regular pics?
If the regular pics produce the same curve, it’s not about the provocative nature of the pics (one of their themes). It would mean something else — like people generally overestimate at the lower ages and generally underestimate at the higher ages.
I think I’m done beating this dead horse. If I didn’t have to worry about my real job, it would be interesting to do the study right.
I’d love a professional sociologist or statistician to analyze this; if we lay people could find so many flaws so easily, how many more are hiding in there?
That’s what I thought, too. The “provocative” part wasn’t really made clear; they claim that it does influence the result (by making younger girls “look older”), but they would need to compare that to results based on non-provocative photographs. Which as far as I could see they didn’t.
The report has too little information. I’m guessing they’re not going to try to write a respectable paper and send it to a journal were people care about statistics. It will probably just end there — and then find its way to a number of propaganda pieces by all kinds of activist groups.
Again, it isn’t what they’re trying to do. They were paid to produce a product, and did exactly that.
All women are 25 years old.
How did I come to that conclusion? Bear with me here, as a little background is needed.
When I was a freshman in college (I was 18) I met a woman who, upon my initial view of her I thought, “she’s might be 16, but I doubt it”. Then she spoke, and I thought “she can’t be more than 13”. She was 19. The last time I saw her, she was 26, and could still pass for 12 (she had a 3 year old daughter at that point). She was a platinum blonde (supposedly, the lighter the color of blonde hair, the younger someone will look because blonde hair tends to darken as we age), extremely petite (she complained about the difficulty of finding size zero women’s jeans, and admitted that she hated that the jeans that fit her best came from the boys department at Sears), with a very high pitched voice. She was about 4′ 8″ tall and maybe weighed 80 pounds, and why she wore a bra I’ll never understand, because it certainly didn’t look like she needed one.
When I was a junior in college (I was 20), I met a woman who, because of her appearance, and because I knew she was taking sophomore level classes I had previously taken, I assumed she was my age or a year younger. It turned out she was 32 (she had to produce an ID to convince me). She was a bit above average in height, and a brunette.
Based on these incidents, and many others like them, I came to the conclusion that I suck a guessing women’s ages. I related the above two stories to a female coworker in 1990. She told me that I should always guess a woman’s age as 25. Her rationale: if they’re older than that, they’ll be flattered because you think they’re younger than they are; if they’re younger than that they’ll be flattered that you think they are “more mature” than they are.
I always listen when women give me advice on dealing with women. Hence, my assertion that all women are 25.
All levity aside, knowing how well I guess the ages of one group of women tells you nothing about how well I’d guess the ages of another group of women, particularly when guessing from photos rather than in person.
Look at the simplest contrary case. I guessed “Sally’s” age correctly. Then I guessed “Susan’s” age incorrectly. Why was I correct in one case and incorrect in the other? Is this something we can even know, based on the sample? Does this tell you anything about my ability to guess “Sara’s” age?
Ask me to look at a group of photos of women whose ages are known, and to guess at their ages relative to 18 you will find out the following: how many women in that group do I think are = 18. Knowing that I am correct about them X% of the time for one group of women tells you nothing about my future ability to do this with another group of women whose ages are unknown. For one thing, unless I’ve correctly guessed ALL the women who are >= 18, you have no reason to expect me to NOT guess incorrectly that some % of women are <18 from a group of photos of women who are all 18.
For instance: you show me 100 photos of women, 50 = 18. I guess 10 of the >= 18 are = 18 year olds, I would guess 20 of them were < 18 when they were not.
All you're really determining in such a "study" is what percentage of photos of a group of women have features that I will correlate to < 18. Knowing how often I am correct tells you nothing useful in predicting my future performance UNLESS you can accurately determine what characteristics I am using to make my determination. And even then, your methodology can result in hideously wrong results.
Pretend the primary characteristic I look for to determine age relative to 18 is breast size; smaller looking breasts = younger. You show me one group of photos, and determine what percentage I am correct. You show me a new group of photos, but this one has a different distribution of apparent breast sizes, and so my guesses have a different distribution (note, by "apparent breast size" I am referring to how a very short woman with B cups may appear to be bigger busted than a very tall woman with D cups; or how a woman with a wide shoulder and rib cage will appear smaller busted than a woman of the same height and cup size, but narrower shoulders and rib cage). But unless the characteristics I look for are actually accurate in predicting age (and as far as I know, there are few surface characteristics that will do this at these ages, although you can tell pretty well if you are looking at their bones), all you're doing is tracking my preconceptions, which may have no relation to reality.
As Maggie says; the Shapiro group is selling something, and since it isn't a product (where false advertising laws would be relevant), they're able to assert "accuracy" where there is none. And certainly, there's none to be found in this study.
Sorry about the length of my previous post, and oops, there’s quite a few places where I had used the greater than, less than, and equal sign that apparently were taken as incorrect hypertext, so a lot of the text where you see the number 18 makes little sense.
Aside from all the other flaws listed above, unless the known-age group were also escorts posing for advertisements, it’s nearly certain that the two populations differed in poses, cosmetics, and photography. A “provocative” pose added years, but any successful prostitute will know far more about provocative poses than nearly any amateur. Most women use cosmetics to try to enhance their appearance and disguise their age, but are pretty bad at it; street whores can just smear makeup on, but a well-paid escort needs to learn to choose and apply it well.
Finally, it would be foolish for a good escort to paint her face and have a friend snap a photograph for her advertising. Invest a little and get a professional makeup artist and professional photographer (or if you swing it right, get paid for the porn shoot and get copies of some high-quality shots). With that, you should be able to appear whatever age you want to be. Hollywood often has 50-year olds playing 20-somethings, and 20-somethings playing 12 year olds.
The ones that really make me cringe are the camera-phone pics snapped in the mirror or at the end of an extended arm. Eeeeeeewwwwww….. 🙁
Hollywood often has 50-year olds playing 20-somethings, and 20-somethings playing 12 year olds.
Ah yes: Dawson’s Casting. One of my favorite subjects (to bitch about). Part of the reason MTV’s new series Skins is so controversial: they have real teenagers playing teenagers. Hey, it’s one thing to have a couple of twenty-two-year-olds (who are pretending to be sixteen) kissing and rubbing together and saying how horny they they are, but REAL sixteen-year-olds doing exactly the same thing!? Horrors!!
BTW, Skins is an American remake of a British show. Other British shows which have been re-made in the USA include Sanford & Son, Three’s Company, and The Office (Steptoe & Son, Man About the House, and The Office, respectively).
Don’t forget All In the Family, a remake of the British ‘Til Death Do Us Part.
Really? I bet my sweetie knew that. She knows a lot about that show.
Indeed. I’m now thinking one should think in terms of populations of cues (poses, cosmetics, photography angles, facial expressions, etc.) rather than populations of photographs. That would make the difference between the two groups even clearer: the cues are very likely to be distributed in very different ways in each group.
You know, there’s been something gnawing at the back of my brain, and I haven’t quite been able to put my finger on it. Something about identifying the age of girls in pictures and CNN and something… something…
We’ve talked about how they can’t really be sure, about how the two groups might not be similar enough, and so on, but there’s something else…
I was listening to Maggie on that podcast and she mentioned how Amber Lyon (whose name I keep confusing with that of 80s porn star Amber Lynn) had put that ad on Craigslist or BackPage or where ever, and she mentioned that Lyon had used a picture of herself when she was fourteen years old.
B-I-N-G-O and Bingo was his name-O! Even if the people asked to look at ads and guess the ages of the girls in the pictures were able to guess with 100% accuracy, every time without fail ever, they still wouldn’t know if the girl who placed the ad was of legal age or not. The girl in the picture could be underage, but the picture could have been taken a decade before the ad was placed. Really, how do they know that the escort isn’t pulling an Amber Lyon?
Thanks, Amber darling, for showing everybody one good reason that Shapiro’s little game here is total BS.
Wow, you’re right! Lots of girls use old pictures, and lots of “hobbyists” complain about it. But I hadn’t considered the possibility that a girl might use a picture of herself from when she was underage! That, I’m afraid, is a blind spot due to technological change; my first nudes were taken with a Polaroid when I was 17, but today’s young whores might have a whole disc full of digital nudes taken when they were 15 or 16. Good catch, Sailor! 🙂
Fourteen year old Amber was in a bikini. She looked good, too. 😀
I bet Shapiro’s age-guessers would’ve pegged her as seventeen.