So far, all of my guest columnists have been female; this month I have the pleasure of presenting my first male guest columnist (other than my husband). Kevin is a graduate student in epidemiology at Dalhousie University in Nova Scotia; he’s also a professional research consultant, which means he gets paid to help people design and conduct their research. I thought you might enjoy reading what an actual scientist had to say about the state of sex work research.
I first came across The Honest Courtesan a little over a year ago after Maggie’s stint as guest-blogger for Radley Balko. Some of the first columns that grabbed my attention were on the topics of bogus research on sex work and baseless numbers feeding malicious public policy, for obvious reasons. Initially I was unconvinced that the field of sex work research was as bad as all that; “Surely, these are just a few hand-picked, exceptionally bad examples”, I thought. I did a review of studies completed to date here in Canada, which has since branched out to the international literature as well. To my surprise, Maggie’s characterization was spot on; studies on sex work often suffered from major flaws that made it hard to take their findings seriously. This was even more surprising considering the fact that research on sex work was no new field; it’s been going on for decades.
Research on a topic rarely starts with expansive, large-scale studies that answer all of our questions on a subject in a single salvo; in young fields, small-scale, low-cost studies help bridge the gap between idle guessing and establishing the first facts about an under-investigated topic. As time goes on, the facts generated from such research are used to develop new hypotheses, which are then tested with more elaborate and rigorous methods. What starts with case studies or focus groups eventually develops into large-scale cross-sectional surveys to accurately portray the population, cohort studies that follow participants for years at a time to establish cause and effect relationships, and eventually systematic reviews and meta-analyses that objectively review what is known about a given topic. As a field adopts increasingly rigorous methods, we can have greater confidence in what we know about the subject. Unfortunately, research on sex work has shown a staggeringly slow progression up this hierarchy of evidence.
One of the first things I noticed when surveying the literature on sex work was the overwhelming number of opinion pieces and editorials, which normally make up a tiny portion of articles published in academic journals. In comparison, authors can routinely be observed arguing for shades of decriminalization and prohibition of sex work in the pages normally reserved for such tedious things as methodological advances and results from new studies. The politics of sex work intrude into the realm of science to an astonishing degree, and I think this is one of the two main causes of the slow rate of advancement in sex work research. As Maggie has pointed out more times than I can count, one side of the debate on the legality of sex work has a vested interest in portraying workers (and clients) as inherently dysfunctional; mass-producing hatchet jobs draped in academic garb is simply a means to that end. Many such “studies” are so poorly designed that anyone picked off the street at random could pull them apart in minutes; they’re a slightly more elaborate version of asking someone whether they’ve stopped beating their wife. The end result is that the academic literature is a minefield of guesswork and kabuki dance posing as, and hidden amongst, earnest attempts to better understand the industry and the people in it.
Aside from the counter-productive dishonesty in the field, the other major limiting factor on the quality of sex work research is that it is really and truly hard to do. The criminalization of sex work in many jurisdictions around the world makes recruiting large numbers of sex workers extremely difficult. Likewise, the social stigma, potential for police to seek workers’ confidential data, and the risk of encountering an unethical quack all likely act to chill sex workers’ willingness to trust researchers enough to participate in studies.
The difficulty in recruiting for studies limits the options of honest investigators. Here in Canada, sex work studies with fewer than 100 participants are par for the course; even recruiting fewer than 50 was relatively common less than a decade ago. This is important because the size of a study’s sample determines how precise the estimates the study generates can be; the more participants an analysis includes, the smaller the margin of error for the results. To put it another way, investigating a population with small samples is like looking at something out of focus; you’re sure something is there, and may even be able to make some educated guesses about what you’re seeing, but the fine details elude you.
Worse still, samples are almost always made up primarily or entirely of street-based “survival” sex workers recruited from addiction rehabilitation programs, which are used as convenient points of contact by researchers. This method means that what is known of the industry is heavily biased towards the most visible and most marginalized segment of the market; the segment where rates of violence, substance abuse, and early age of entry tend to be highest. The low-profile indoor market, which is generally estimated to make up ≈85% of workers (though we have no hard data with which to confirm this in Canada), is vastly under-represented in the literature. Even when indoor workers are included in studies, the sampling hasn’t been designed to represent the sex worker population as a whole, meaning that even our most detailed picture of the industry is blurry and incomplete.
Despite everything you’ve just read I’m still optimistic about the future of the field for two reasons. The first is that the composition of the people doing the research has changed a lot in the last thirty years. While the appearance of HIV/AIDS was a huge political setback for the sex workers’ rights movement, it does appear to have attracted a new group of researchers, and much more rigorous methods, to the fray. My review of the literature leads me to believe that, prior to the 1990s, most work done in the field was in disciplines such as Sociology, Criminology, Political Science and even the dreaded Women’s Studies; terms like feminism and patriarchy can be seen in actual, straight-faced research papers. Epidemiologists and public health researchers began entering the field at some point after that, often tackling sex work research by investigating sexually transmitted infections, addictions issues, and the effect of public policy on workers’ health and safety. Instead of engaging the topic from a political perspective, most modern sex work studies have adopted the dry, technical language and emphasis on coherent, replicable methods that one might expect from proper research. Harm reduction has long-replaced feminism as the dominant framework seen in the literature. The changes haven’t merely been superficial; whereas the studies of thirty years ago recruited <50 participants in one-off focus groups, some modern studies track over 500 sex workers for years at a time. In countries where HIV/AIDS is more endemic (e.g., India), studies recruit workers by the thousand. In short, researchers in the field of sex work are now putting in the hard work needed to get serious, credible answers to the most basic questions in the field; this isn’t the only recent change, though.
The second cause for optimism is the ongoing development and adoption of methods that enable researchers to circumvent some of the barriers to large-scale and representative recruiting of workers for studies, and for the first time put sex work research on par, quality-wise, with other fields. Even the hardest-to-reach participants in the industry (e.g., underage workers) can now be efficiently recruited into large, representative samples that allow for accurate estimates of population size and structure; for the first time, we can credibly answer basic questions such as ‘How many sex workers are there?’, ‘How many work indoors versus outdoors?’, and ‘How many enter the trade underage?’.
It’s probably fair to say that, historically, the sex workers’ rights movement and the academic study of the industry haven’t had a good relationship; much of the time the field has seemed to be implicitly working for the other team. Nevertheless, as the objectivity and quality of research improves, and as we learn more about the industry as a whole, fabricating “facts” about sex work becomes less and less effective a tool for pushing dangerous, wrong-headed policies. As was the case when Canada’s sex work laws were tested in the Supreme Court earlier this year, the ability to decisively cut-down prohibitionist myths in real-time and in front of a panel of judges can be a powerful tool in fighting for sex workers’ rights.
Here in NSW it was the big breakthrough.
HIV plus the fact that police corruption had become so blatant and endemic even the mainstream media could no longer ignore it.
I’d be interested in knowing in which fields of research Kevin Wilson thinks there is good honest statistical analysis going on.
Some fields I have been involved in (as a student, advocate and/or lecturer), some common statistical tricks they employ and the skews they give.
Drug addiction and sex offender rehab programs
– Eliminating drop outs from the final figures. (Makes more onerous treatments look better by only counting those with strong incentives to complete.)
– Self reporting follow ups (if you’ve gone back to shooting smack or molesting kids you’re not likely to phone back and tell the researchers about it)
Pharmaceutical testing
– Surrogate endpoints and data slicing (come up with over twenty different ways to look at the data and its not hard to find one with p<0.05)
– No placebo controls for 'ethical' reasons (you don't give sick people a placebo if there is already a treatment, so if you compare one SSRI with 64% efficacy against another with 67% efficacy you don't have to note that the depression index you're using gives sugar pills 65% efficacy)
– Chronic or cyclic illness and 'the law of thirds' (if you test a group of people with a non-degenerative illness then test them again a bit later a third will report feeling better, a third worse and a third 'no change' – viola you've got 33% efficacy before you even begin treatment)
Forensic science
*Sigh*. Where to start? I could write over a dozen papers on this. In fact I have.
– Applying the product rule to dependent factors and the prosecutor's fallacy (see Meadow’s Law for good examples).
– Using population statistics while ignoring human error rates (The DNA match probability is one in a trillion? A shame that accreditation testing shows lab error rates of around 1.2%, so we won’t mention that)
– DNA database cold hits and multiple crime scene samples(We’ve just got a one billion to one match on this guy. But we ran twenty crime scene profiles against three million convict profiles to get it. i.e. the ‘real’ match likelihood is one in 16.7, but thanks to Paul B Ferrara and Bruce Budowle you won’t hear that in court)
This comment is already way too long, but I’ve got many other examples and several other fields I could write about (or have written about), believe me.
id like also to know how they find theese facts about pornography.in a recent campaign when they ”nicely ask the head of a certain british tabloid to take away topless images in the paper”they say that boys who might see that grow up to think that girls are a fair game.
also, such people make non logical arguments often and yet are taken seriously which drives me nuts more than anything. for example in that case they say that its not a free speech issue since topless images dont speak(pure sophistry,i think)and they are taking liberties since they reinforce a narrow ideal of beauty and send the message that if youre not desirable you dont matter.however,lets just say that is true and someone through their media platform sends such a message(like Karl Lagerfeld who said that life is no good if youre not thin),how are they actually taking liberties?its still up to people to take them seriously,if we take it for granded that teens wont be able to say that thats pure bullshit its up to the parents to teach them certain things.we cant just ask or most correctly pressure for the removal of every opinion we dont like.that is taking liberties.the only way for a person like Lagerfeld who wants to influence people to be thin to take liberties is if he used a hypothetical influence he had in the media,so as not to publish opinions that go against what he wishes to acheive.basically what politicians do with the help of their lapdog channels.what makes me wonder what people have in their heads is not when they beleive false statistics,its when they support opinions in terms of sex or sex work that in any other topic they would probably call out as bullshit.
While biased studies show up in any situation where people want to legislate based on their feelings, in a lot of cases large, profitable industries operate to act as a counterweight to the biased studies. For example, the same people who work hard to keep sex work illegal and put in ever crueler penalties want broad based censorship across all sorts of media. To justify this, they will put out biased study after biased study, using previous biased research as sources for “new” reports. However, they can only get so far before they start to cut into profit margins, so commonly you get two groups of compromised studies a pro and an anti in any situation. This, of course, leads to the situation we have now, where studies are basically just rhetoric to be deployed the same way one would a charismatic speaker.
This is why, in general, I must unfortunately reject “rule by study.” Conceptually this is the basis of “technocratic” rulership. In which the idea is you do a study, after you get the results and legislate based on them, and thus ever improve society toward utopia. Much the way Stalin used quack science to justify certain of his more destructive programs around winter wheat.
So, if rule by study is out, what is left? Rule by principle. So, in my case, that would be based on the principle of self ownership. If you choose to harm yourself, we should allow it and only concern ourselves if you harm unwilling parties.
The problem of course is that we still have a few exceptions to self-ownership based around the concept of insanity. In other word, if a person fears being devoured by spirals, and decides to cut off their fingertips, I want to stop them because they are making decisions based on a delusional outlook.
Unfortunately, the opposition has also caught onto this and pushes various forms of mental health arguments why violent or sexual media or unconventional sexual activities are either signs of or contributors to insanity. Currently, the accusation of madness seems to be one of the more powerful arrows in the tyrants quiver (Stalin routinely locked up dissidents he didn’t want to kill for some reason in madhouses until they “got their minds right.”) and it’s this area where countering it is difficult. The only option, it seems to me, is to set a really high bar for insanity before you deny a person self-ownership.
So, I tend to reject rule by study out of hand and set a very high bar before I’ll accept false consciousness as an excuse for stopping a person from making their own decisions. (for example, if a person believes they’ve been abducted by aliens, but doesn’t do anything to harm others, I won’t argue in favor of involuntary committal, even though I severely doubt his story.)
Oh, and that doesn’t mean that I don’t believe in deploying studies which bolster self-ownership. By all means, use whatever weapon you’ve got! It just means that even if there were no studies favoring self-ownership of an individual’s life over State ownership of the individual’s life, I would still reject the pro-State ownership argument.
I am not a number, I am a free man.
Couldn’t have said it better myself. 😀
I don’t trust any study – even the ones I like. Proper studies cost money – and the only people willing to really fund them are people with an agenda.
Great post! Kevin, I am curious about the scale and scope of the studies you perused – do you have a biblio that Maggie could offer us? I think one of the best points, IMHO, was the use of language to obscure ideological intent behind the ‘study.’ Who is not for ‘harm reduction’? It can swing in several directions.
What you have done with modern research on this topic, I am now doing with the historical record: the prejudicial commentaries are just as glaring once the facts are traced and verified.
‘Rule by study’ does seem to privilege the elitist – I know of one patron willing to bankroll a study that will further her agenda on Demand. One study was already done and some Harvard economists had real trouble with its methodology. This did not keep her from continuing to fan the flames of hysteria to promote her agenda. In a way we need ‘Snowdens’ in the anti-trafficking movement to be whistleblowers once they have an awakening as to the shoddy ‘evidence’ the public is being fed.
Thanks, Kevin for the précis on sex-work studies. It was very illuminating.
Evidence based medical practice really took off with the Cochrane Collaboration; before this we only had “expert opinion”, most of which has been shown to be unreliable at best.
A major problem with drugs trials has been non-publication. Drug companies organise, fund and sometimes write up the results of trials. Unsurprisingly, perhaps, those trials which show positive results are the majority. Trials which show the side effects — sometimes (potentially) fatal — don’t (always) get published.
It wouldn’t surprise me at all if studies of sex work whose results don’t fit with the beliefs of the investigators aren’t published.
Archie Cochrane’s self-written obituary must be one of the most candid; I particularly like the last paragraph:
http://www.cardiff.ac.uk/insrv/resources/scolar/bmj_alc_obituary.pdf
(A merit award is an extra payment to NHS consultants, given for services above and beyond the “call of duty”.)
So, who’s going to do such a collaboration for sex workers?
Excellent post! And thank you for your professional work in the field.
Just a footnote here:
Luckily the California Supreme Court forbade that particular statistical lunacy long ago, People v. Collins, 68 Cal.2d 319 (1968).
My reading of the case is that the prosecutor was actually so stupid or ignorant that he actually believed it was valid.
Kevin-A great column.
Searching for the truth in today’s era of easily manufactured news is a pain in the neck. And some days I have an even lower opinion of it.
I would be interested in your opinion on my OpEdNews article from March of last year “Making Sex a Crime.” I tried my best to separate fact from fiction, and I would be very interested in your opinion of how I did, especially because I am in the process of putting together a follow-up article.
Thank You in advance.
[…] Guest Columnist: Kevin Wilson (maggiemcneill.wordpress.com) […]
It was nice to hear from somebody involved in research studies, somebody who knows what’s what in that world.
[…] a year ago a fantastic guest column was posted on the Honest Courtesan. I have a passion for research and science, and it frustrates me […]
[…] credible) findings concerning…gonorrhea rates and arrest rates in the state,” writes Wilson. And while that may be true, I feel that those findings’ inclusion in a study with so many […]