Closely Observed – Finding and Using Tertiary Material

This chapter is designed to help you understand what tertiary materials are including scientific publications, field diaries from participant observation, as well as tables and stats that can be obtained when looking at sources like census data. You will learn how to locate, find and credibly read tertiary documents while acknowledging that there are limits in how this information can be used in your research project. Finally, you will be instructed on the necessary steps to engage in participant observation if your research requires it.

1. Understanding Tertiary Materials

  • Tertiary materials are intended to PROVIDE INFORMATION and in and of themselves, offer no explanations, no theory and very little explicit analysis
  • Examples include tables and graphs as well as the census info, large survey data, and statistics often found in government reports
  • Tertiary data also includes the raw data (research) in natural and social science papers and even the references and footnotes in bibliographies and newspapers
  • Finally, field notes gathered in participant observation can also be classified as tertiary documents
  • Participant observation refers to systematic observations and note-taking on site and for a specific period of time in connection with situations such as community interactions, meetings, and conferences (more details below)
  • These observations are usually recorded in something called a FIELD DIARY which involves a specialized form of writing designed to capture observations in a way that allows the researcher to use these observations later when writing up their research
  • It is the opposite of action research where the emphasis is on participation as opposed to observation
  • The key issue to understanding tertiary documents is credibility – while you may want to use this data as evidence for your own research, you must not misread what is in the original documents which is a common thing to do, especially when it comes to interpreting scientific materials

2.  Reading Science

A. Interpreting Scientific Materials

  • Remember that scientific writing is tertiary insofar as the findings from scientific observations or applications from scientific techniques (e.g. surveys, experiments, modeling) are concerned
  • Despite the fact that scientists often talk about their work as being “objective,” it is important to understand that scientific data rarely provides definitive results and no research is ever entirely free of perspective
  • Instead, think of science as a body of strategies used for systematically observing, learning about and explaining phenomena
  • Different scientific disciplines may have different strategies (e.g. theories vs. case studies vs. observational data)
  • Before you report an any research, you must first establish its credibility, and in the social and natural sciences, this is done in two ways: a) any study is discussed in the context of other studies, as part of a body of material that is subject to debate but is otherwise connected, and b) emphasis is placed on the methods used to collect and analyze information
  • This means that credibility is essentially established by INSIDERS (i.e. those already familiar with the kind of research and topics under study)
  • Ways to properly assess scientific materials include determining reliability (a term used to indicate other researchers focusing on the same problem and following the same methods would reach similar results), validity (indicates conclusions are sound and properly reflect what has been observed and/or calculated), and probability (based on statistical analysis and used to indicate the degree to which research findings are not likely to be due to chance)

B.  Common pitfalls when reading science

  • While it is perfectly reasonable for a non-scientist to read scientific materials and draw conclusions, you still need to avoid certain pitfalls if your own discussion of science is to be credible

i. Not recognizing the obvious

  • Since scientific findings come from many fields of study, you need to understand the methodology and conclusions in relation to the specific discipline itself
  • Often, this means you will not know how to read the studies of different disciplines, and the solution here is to rely on science journalism as opposed to original research or to consult someone knowledgeable in the discipline

ii. Don’t rely on one single study

  • In the sciences especially, information from a single research study is rarely useful on its own
  • Usually, you need multiple studies or corroborating research to produce information considered useful and trustworthy

iii. Don’t depend on anecdotes and intuition

  • Don’t use anecdotes or intuition as sources of data as science often produces results that seem counter-intuitive
  • While anecdotes and intuition may draw attention to ideas that might otherwise be ignored, in and of themselves, they do not produce credible conclusions.

Question:

I am a bit confused. What you mean by anecdotes and intuition?

Answer:

Sometimes, we are prone to draw conclusions based on single story or on information that seems obvious. For example, the members of a community located near a power plant make observations about the rising number of cancers in their area. On the basis on these observations they intuit that the emissions plant is responsible for this excessive number of cancer clusters. Well, while community members should definitely following up on their intuition, it might be the case that the cancers and emissions have nothing to do with each other. One would need a much more detailed analysis and sophisticated statistical techniques to know if their intuition was right.


iv. Know the difference between probabilities and certainties

  • Science is always more or less uncertain and this means that most natural (and some social) sciences trade in probabilities as opposed to absolute certainties
  • This does NOT mean that once scientific conclusion is as good as another; it just means that while some conclusions may be stronger and more reliable than others, they are rarely uncontroverted
  • Probability is a common measure in a lot of scientific work and probability estimates are based on the idea that if left to chance, there will be a clustering of results around the middle or average with fewer results at the more extreme ends
  • Usually, a study is considered noteworthy because the findings deviate from that which would be expected if the result was simple left due to chance

v. Lots of things can’t be proven with science

  • Many things cannot be “proven” by research in the natural and social sciences
  • For example, lots of chemicals we have banned day were initially considered “safe” for public use, and this change in status was only possible thanks to new research studies which presented new evidence that was absent before – in other words, science can’t always provide us with definitive conclusions no matter how important these issues are to the public

vi. Be careful about mapping one research study directly onto another

  • Despite the fact that scientists often talk about replicating research, it is rare for the detailed findings of one study to directly map onto the findings of another study
  • This means that there are usually differences between one study and another and the importance of these differences is debated whenever conclusions are being drawn

C. Using findings from scientific studies

i. Find science material of interest to you

  • There are many good science databases (aggregators) that will help you obtain useful articles which contain the findings of scientific work
  • Examples: Web of Science, Science News, Scintilla, Science Daily, Science Magazine, Reddit Science, Research Blogging, Science Seeker (links to websites required here)

ii. Find science journalism that deals with your public issue

  • Sometimes, you will find it far more useful to consult science journalism materials as opposed to original research articles
  • In science journalism, information is summarized, the points of controversy noted and the significance of the material is discussed
  • This means that scientific journalism tends to give you a broad overview of topics without many of the details found in hard science journals
  • You can find science journalism in places like the science section of a newspaper or in general news (e.g. Time, Newsweek, The Economist) or scientific magazines (e.g. National Geographic, Scientific American, Science, Nature, Popular Mechanics, Physics Today)

iii. Consult Professional Publications

  • Many professional organizations often report on scientific studies
  • Examples include Institute of Electrical and Electronics Engineers (IEEE), Canadian Medical Association, American Medical Association

iv. Consult someone knowledgeable

  • Even with diligence, there are limits to what you can learn about scientific research, especially if it concerns a discipline you know little about
  • Consulting an expert in the field (even with a specific point of view) will allow you to become more aware about the uncertainties and points of debate about the topic under consideration

3.  Understanding Participant Observation

  • While participant observation provides a richness of information likely unavailable from any other method, there is no way to avoid the heavy imposition of values, preconceptions and personal predilections of the observer

Question:

I find the idea of participant observation fascinating but how to do I ensure my own observations aren’t clouded by my biases, values or personal beliefs?

Answer:

Those who do participant observation have adopted strategies for handling the problem of observer bias and predisposition. These include: a) lengthy and different periods of observation; b) identification of questions prior to research which predispose the observer to see what otherwise might be missed or discounted; c) using a field diary that include both situational and self-observations; d) searching for corroborating materials such as documents and interviews; e) having the observer play “devil’s advocate” which involves seeking evidence of counter-intuitive insights and observations that fail to confirm expectations and e) using a “triangulation” method which involves locating different points of observation such that different perspectives can be brought to bear.  


Research Steps

i. What situation is worth observing?

  • The first step in participant observation is to determine which situations are usefully observed in person
  • This might include situations officially designated as deliberations such as meetings, legislative committees, inquiries and regulatory proceedings, trials, conferences, public events but may also include places like clinics, libraries, university classrooms or even a hospital waiting room
  • Depending on the situation, you may be able to witness power relations, institutional functions and the purpose and role of the setting itself

ii. When to get consent

  • Next, you must get consent for observation – this is essential if the situation is not one that is open to the public
  • In public situations, the guidelines for getting consent are less clear and somewhat impractical, as it would be difficult to obtain consent from everyone in a hospital, university classroom or public hearing
  • In these cases, while you still need permission from someone in charge to engage in your research, you don’t need to obtain consent from everyone involved if no one will be identified personally and no one will be directly quoted
  • In situations where people are making on-the-record formal presentations (e.g. legislative assembly or council meeting), you do not need consent as participants expect to be observed and recorded
  • However, if you do intend to use their comments for your research, you should get consent first
  • In terms of casual conversations, if the person you speak with is unidentifiable and no quotes are used, then the conversation is simply a part of your observations – if not, the conversation then has to be treated more like an interview (see next chapter)
  • When in doubt, OBTAIN CONSENT!

Question:

Do I need to obtain consent if I take someone’s picture?

Answer:

If you take someone’s picture and intend to make use of that photo in your research, you need to obtain that person’s consent first. However, if no individual can be personally identified from your photo and you seek no commercial gain from it, you do not need consent although it is courteous to seek it.



iii. Prepare before observing

  • Make sure to collect as much material as possible before starting your observations
  • This can include items such as agendas, speaker and/or participant lists, background papers, CVs, past media coverage, and meeting rules and practices

iv.  Visiting ahead of time

  • Try and visit your place of observation ahead of time
  • This will allow you to draw a rough map of the arrangements of the room
  • This is important, as often, where the table and chairs are placed can say a lot about what is supposed to happen

v.  Draw up a list of questions

  • It is very important to draw up a list of questions ahead of time that can be answered through your observations
  • The purpose of these pre-determined questions is to allow alert yourself about what should be observed closely
  • These questions will provide you with guidance about the most important aspects of the situation for your research and will also allow you to challenge whatever preconceptions you have going into your situation

vi. The observation process

  • During the observation process, try and sit as far away from the main stage of action as possible without it impeding what can be observed
  • It is better for you to not participate actively or draw attention to yourself as your goal is not to observe others’ reactions to you
  • While it is important to focus your attention on what is being said, you also want to look around and considered other things such as “who isn’t here?” or “what isn’t being discussed?”
  • Your answers to these and similar questions are part of the your observations as well
  • Other things to consider include observing body language as well as the expertise and interests of those you are observing
  • While you want to refrain from making stereotypes, you may also want to guess at the class and status of participants, and this can include questions such as “Is diversity evident in this situation, or do the people all look the same?” or similarly, “what kinds of people have been invited to speak and what elements are included in their presentations?”

Question:

I am feeling a bit overwhelmed by the idea of making all of these observations. What if I miss observing something?

Answer:

We understand that the list of possible observations is very long, and we don’t expect you to observe everything. After all, paying attention to one thing will mean not paying attention to something else. For example, if you are looking at what is going on in front of the room, it means you can’t see what is happening behind you. Again, that is why having a well-thought out list of questions beforehand is important, as it will go a long way to making sure your observations are sufficient for your research. In fact, you will probably only observe about 15% of your situation in total. That said, limited as it seems, your observations are a crucial element in your research, and what you see is at least as important as what someone else says or what is stated on a document.


vii. Note-taking (Field Diaries)  

  • The notes you take during participant observation are called a field diary
  • These notes should be no more than jottings such as a single word, phrase, scribble-type drawing or shorthand symbol in addition to exclamation or question marks in the margins
  • Your notes taken during the event will be incomplete but that is okay because if you spent all your time taking notes, it takes your mind and focus away from the actual observations

viii. Filling in your Field Diary

  • Once you have completed observing your situation, immediately go to a quiet place and fill in the details of what you saw and heard
  • You will be surprised at how much you recall, but this is only true if you ensure there is a very small window between your observations and completing your notes
  • If you go for lunch, check your email, or text a friend in-between, you are creating distractions which will impact your recall abilities
  • You need to allocate several hours after each observation session to expand on your notes and jottings

ix. Add “notes to myself” in your field diary

  • In addition to your observation notes, don’t forget to include your personal comments, judgements or feelings about what has been said or observed in your field diary
  • It is likely you will remember your own personal thoughts and judgements even more than your observations and this will help lead towards analysis in your research
  • However, make sure to demarcate these personal comments to distinguish them from your observational data

x. Tie observations to interviews

  • Make sure to tie your observations to your interviews, as your interviews will fill in many gaps and act as a check on the undue influence of your personal reflections

4. Understanding Census Data

Steps in using census data

i. Find the data

  • Most countries maintain statistical bureaus that carry out a census of their population and provide you with an overview of industrial activities in their jurisdiction
  • In Canada, you can find census data through Statistics Canada (LINK REQUIRED)
  • You will find a wide array of information ranging from topical census data to community related information such as housing costs, income, language usage and population
  • To access detailed census data in Canada, go to one of the depository libraries (e.g. university, legislative or select public libraries)
  • that receive all Stats Canada and/or Federal Government publications
  • Census data from other countries can be found through the following links: US federal government (LINK REQUIRED), Britain’s Office of National Statistics (LINK REQUIRED) or National Archives (LINK REQUIRED), Nigeria’s National Bureau of Statistics (LINK REQUIRED), Brazil’s Insitituto Brasileiro de Geografia e Estatistica (LINK REQUIRED), or China’s National Bureau of Statistics of China (LINK REQUIRED)
  • Depending on what country you are dealing with, a local consulate, embassy or trade office may be able to provide you with census data and reports in addition to academic and business journals and various institutional websites such as the United Nations, World Trade Organization, OECD, World Health Organization, International Monetary Fund and World Bank (LINKS REQUIRED)

ii. Carefully examine the data

  • While the data or table stands on its own as a representation of information that has been amassed and organized you need to look at what is really being said so that you don’t read into the data more than you should
  • The tables you find in census data and elsewhere summarize information and present it in such a way that comparisons can be made
  • In the text surrounding a table, conclusions are usually drawn, and data are organized so that you will see how and why these conclusions are justified
  • Read the titles carefully as far too often, tables are presumed to say one thing and actually say another, or the info in the table is applied to situations that are not depicted in the table
  • Examine the notes accompanying the table which indicate the limiting conditions under which info is gathered, the sources of info, and any constraints that need to be factored while reading the table
  • If the table is going to be used in the context of your own study, it is especially important to pay attention to the information sources and the original source of the table itself so that you are confident using the data moving forward
  • This includes going to the original publication to see how the table was originally constructed and asking yourself the following questions: was the publication peer-reviewed? Was the info taken from a reputable source? Was any information censored? Were there any check and balances in preparing or publishing the material?
  • Next, pay close attention to the categories used to organize the information in the table, especially the column titles
  • Column titles represent indicators, which means only some aspect of the phenomena being described has been measured in a particular way to capture the picture as a whole
  • For example, the title “children’s health” in a table might only be represented by the number of visits taken to a hospital
  • It is in the choice of indicators that information is shaped, skewed and sometimes biased and that is why each indicator in a table should be examined to see what is actually being measured or referred to in organizing and making sense of the information

iii. Pay attention to why table was constructed

  • You need to have some understanding about why the table was constructed and what the author was trying to show
  • The title and text of the table will give you some indication about the purpose of the table, and ask yourself questions such as: What is being compared? What has been included and left out?

iv. Extracting info from the table

  • To summarize, before you extrapolate data from your table, have you paid attention to the title, notes, definitions and categories being used?
  • Without understanding all the information surrounding a table, your interpretation of the data will likely be subject to many misunderstandings!

5. Understanding Statistical Information

  • Stats often portray comparisons using numbers that are usually in the form of percentages
  • Stats are another way of representing information and are sometimes quite similar to information found in a table
  • Sometimes, statistical information has a purpose beyond the numbers represented in a table
  • In such cases, conclusions from the data are applied to a larger group or category
  • For example, if you conducted a survey of 100 small and large universities and came to the conclusion that “small universities have a better teacher-to-student ratio than larger universities but less choice in subject areas,” you may have only examined these 100 schools, but the conclusions are being applied to the bigger categories of small and large-scale universities
  • To ensure you use stats correctly, it may thus be a good idea to find someone knowledgeable to consult with who is familiar with the discipline and/or methods being used
  • This will allow you to learn whether the methods and materials are considered trustworthy, or whether there is some controversy with the data itself

Case Studies (participant observation)

The Consultative Committee on IBT Pesticides

We were a motley crew, a pharmacologist, toxicologist and a few other natural scientists, an agricultural economist and myself a social scientist.  It was obvious from the first meeting in the early 1980s that we would understand the issues to be determined by our consultative committee on the safety of particular pesticides in different ways.  What was not obvious, and I could never have predicted, is the perspectives of my fellow committee members.

I came from the discipline of Communications but I had little or no interest in how the pesticides in question were treated in the media.  I knew that there had been a controversy in one of the newspapers but I did not see this controversy as having much to do with the task at hand, which was to make an assessment and recommendations about the pesticides. Several of my colleagues from the natural sciences were almost exclusively focused on the role of the media.  From their perspective, it was the press report, and resulting small-scale controversy that turned attention to what otherwise might have been a routine matter of scientific assessment.  It was the media that had raised the alarm that made our work necessary and useful.  We needed to appreciate both media and their impact on public opinion.

None of the natural scientists made any claim to have disciplinary knowledge from the field of Communication.  In their view, expressed in various meetings, anyone could see what had happened, and speculate usefully on the reasons why.  The media and public opinion were matters that any attentive member of the public could comment upon usefully.  At the same time, we, who lacked disciplinary knowledge in the sciences related to pesticide testing and safety were not in a position to do a scientific assessment of pesticides, no matter what information was brought before us. Thomas Guerin uses the term “boundary maintenance” to explain how communities, scientific disciplines in this case, protect their boundaries against those who claim to have something to say about matters at the heart of the discipline.  In short, while my colleagues considered themselves to be “insiders” to discussions about media and public opinion because thewy were members of an attentive public, the agricultural economist and I were outsiders to the disciplinary communities properly consulted about the issues that the committee was asked to address.

The issue at hand was the revelation, actually some years earlier, that one of the laboratories responsible for testing a number of pesticides had been either careless or negligent in its scientific assessments and conclusions about a number of pesticides.  The matter had been raised by a small town newspaper.  It resulted in a conflict between two Ministers in the Canadian parliament  In a parliamentary system, Ministers are supposed to keep their conflicts behind closed doors.  Now that the conflict was public, something had to be done.  It was not clear whether that “something” was a reassessment of the pesticides in question, an assessment of the regulatory system that had allowed the faulty data to be used and the problem to remain long after it had been discovered, or was it simply that there was a political problem that required addressing.  In any case, our committee was called into action as a sort of mini-public inquiry.  That we had such different backgrounds reflected the various purposes, and so too did our mandate.  Our job consisted mainly of reading documents and holding public hearings in various cities across Canada.

In the end, the meetings had the feel of “horse-trading”.  Each phrase in our final report was fought over:  Would we say these pesticides should be considered safe in light of the lack of evidence to the contrary?  What did it mean to use the term “safe”? Would it make sense to include cautions about pesticide sprayed fruits, posted on the fields where spraying had been done or on a sign in the grocery store?  What, if anything, would it be useful to say about how pesticides were regulated at the time in Canada?  Given that this last question was not specifically raised in our mandate, should we address it at all? In the end, we reached an agreement, reluctantly on some people’s parts, on the wording of our recommendations.  Keep in mind that we were empowered only to recommend.  We had no power to decide anything.

For me, the most interesting moment came in a hearing in Vancouver. We heard there from a group of environmental lawyers who had consulted scientific experts in the preparation of their brief.  That said, being environmental lawyers, the main focus of their brief was on the regulatory system for pesticides, a subject we could presume they knew a great deal about.  We also heard from a representative of the farmworkers union, a small but highly vocal and militant group that defended the interests of the mainly immigrant workers who came into contact with pesticides on a daily basis.. The environmental lawyers spoke clearly, softly and their brief was very detailed, with lots of references. The farmworker representative spoke forcefully. There was emotion bound up in every phrase, mainly anger needless to say.  There was little of no scientific discussion in his brief, and indeed little or no mention of the specific pesticides we were assessing or the regulatory system that had failed to act when the original problem was discovered. A specific proposal was made: posting fields with a sign after they had been sprayed, and the committee was asked to put pressure on the provincial government on worker safety issues, a matter clearly outside its mandate.

I observed that the environmental lawyers looked much like the committee members, the same way of dressing, same demeanor, same race etc.  The farmworker representative was clearly different, not just in terms of class and race but also in his manner of speaking.  Afterwards, we on the committee regrouped behind closed doors to discuss what we had heard.

At the time I was surprised.  The farmworker had a sympathetic audience, notwithstanding the fact that, as I had evidence to support by now, none of the other committee members shared his political perspective.  The environmental lawyers were all-but dismissed.  In the discussions that followed, and indeed in the final report, it was the farmworker’s concerns that were reflected, to the extent our mandate allowed, not those of the environmental lawyers.

I pondered for some time why the environmental lawyers had not been taken very seriously.  I could see why the powerful rhetoric and personal stories provided by the farmworker representative had an effect, but I could not quite understand why the truly competent lawyers with their very much to-the-point brief were not. Finally it dawned on me, and I followed up with interviews later.

The farmworker represented those with an interest, a direct interest in the matters at hand.  The environmental lawyers did not have a discernable direct interest.  They were there because they believed the issue was important, and that they could make a public contribution by addressing those aspects of it about which they had formal expertise.  In short, interests and stories about actual experiences had trumped formal expertise and altruism, so much so that I never heard further mention of the environmental lawyers or their brief in any of the discussions thereafter among my fellow committee members.

When I was asked to join the committee, I made it a condition that I be allowed to observe and write about what happened during the course of our mandate.  All members of the committee were informed, not once but several times, that I had a dual function.  It did not seem to matter.  People were exceptionally frank about their views, regardless of what they may have thought about what I was doing.  I kept my side of the bargain.  With a little research, one could find out who was on this committee, but no one was quoted, and nothing that was deemed confidential was every revealed.  Not only did I publish the results of my participant observations, but I was asked to assist the government officials in several tasks after the committee’s work was completed. I used these same observations to write about more general issues concerning the interplay between science, law, politics and values later.

I kept detailed notes on every meeting.  It was incredibly time consuming and often boring to do so.  I would write down a few lines in the actual meeting, and re-group later to fill in my notes from what I had seen and heard.  I learned a lot from the formal record, and much from my colleagues on the committee who had expertise in the matters we were discussing.  Mostly I gained my insights by mulling over what I had seen, and especially what perplexed me at the time. The notes served as backup, but they were only the starting point for the musing I needed to do.

Liora Salter


Research ethics boards

I am looking at the harmonization of risk policy in research involving humans.  I examine the impact of standardization on the system of ethical governance in academic and biomedical research, as well as on research involving humans itself. More and more researchers in various fields of academic inquiry in Canada and globally are required to “pass ethics”, or in other words, to get approval from a local ethics review board before they can begin their research. In Canada these review boards are called Research Ethics Boards (REBs). The task of these institutionally based REBs is to ensure that research involving humans is conducted according to the highest ethical standard.  I have been participating in one such board for some time.

The purpose and composition of REBs in Canada are defined by the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS). The first edition of the TCPS was adopted by the Canadian Institute of Health Research, the Natural Sciences and Engineering Research Council and the Social Sciences and Humanities Research Council in 1998. This was one of the most significant events in the interdisciplinary harmonization of approaches to ethical governance in research involving humans. In 2001 the three major Canadian research agencies also established the Interagency Advisory Panel on Research Ethics to interpret and further develop the policy. In December 2010 the Councils adopted the second edition of the Policy which affirmed and solidified the approach at harmonization.

The objective of standardization was to ensure uniformity of the law and consistency in its interpretation and application. In theory, this should facilitate further integration of different jurisdictions or fields and lower administration and transaction costs. In theory, a better standard should be chosen as a basis for harmonization, but in practice it is not always the case.

The Research Ethics Boards’ review follows a  biomedical approach.  The biomedical practice of risk management is attuned to the biomedical and behavioral understanding of research and the risks of harm that it may pose to human subjects. However, when this practice was extended to the social sciences, it generated a number of problems and legitimate concerns about the ability of the biomedical model of risk management to succeed in achieving its objectives across the whole disciplinary spectrum.

The biomedical standard of risk management is a product of a particular understanding of research and risk. It also comes in package with a particular conceptual framework, on which it is based, and a heritage of ethical problems. Although the requirements of free and informed consent (preferably paper based to better manage legal risks), anonymity and generalizability of data, understanding of research as a sequential and systematic endeavor, as well as the focus on individuals rather than collectivities, may seem fit and have shaped an understanding of ethical conduct in biomedical sciences, they appear to cause serious tensions when transplanted to the field of social sciences, which feature a broader spectrum of research methods.

Research Ethics Boards are often criticized for being secretive, opaque, and formalistic bureaucracies. Yet they do allow observers to attend their committee meetings.  This is where I came in.  I observed the discussion of proposals that were deemed to pose more than minimal risks to human subjects take place.

Participant observation is one of the methods of social research, especially popular among ethnographers, but practiced universally and often without acknowledging its place in the repertoire of used methods. Participant observation of organizations dealing with professional governance is important as it lets us study a dimension that would be difficult or even impossible to examine by other methods of policy research. Surveys, interviews, focus groups, document and content analysis although effective in studying material resources and technologies of an institution, may be less effective in examining institutional culture, its modes of thinking and acting. There are at least two reasons for this. First, there may be a difference between what actors do and what they say they do. Second, actors are plunged in their own culture that may limit self-reflexivity.

Participant observation helped me in understanding why Research Ethics Boards stay away from a localized interpretation of general ethical principles and keep searching for external “best practices”, rather than engaging in development of an internal “best practice”. It also allowed me to register why Research Ethics Boards tend to focus on the formal aspect of ethics review, that is, on procedural rule-following, protocols, consent forms, quorums, and revisions, and reporting, rather than on persistent ethical problems, such as ghost writing, which pose immediate risk of harm to human participants.

The seemingly antithetical nature of participant observation as a research method had a number of advantages. Indeed, there was no sense in trying to withdraw from the situation and serve as an impartial and removed observer. The whole idea was to participate and observe the situation and myself as one of the participants. This gave me an opportunity to feel through the ethics review procedure from within, to experience its pressures and limitations.

Introspection, if done critically and systematically, can be a powerful method of social research. A participant observer has to be cognizant of the fact that he or she is contributing to the situation. In pure observational studies researchers try to exert as little influence on the situation as possible, whereas in participant observation this may undermine the participatory component of the method.

If you are a researcher who uses participant observation as a principal research method, you will likely experience problems in passing ethics review. The method itself appears to be ethically problematic for Research Ethics Boards. This is another reason why I am interested in participant observation. Participant observation goes against the very ideology of the current risk policy in research involving humans. Therefore, it helps to identify the points of tension in the system of research governance.  Indeed, REB members may find participant observation to be insufficiently objective, which on its own can be taken as a sign of poor ethics. The Board will say that the results may not be generalizable and the capacity of participatory research to be systematic is questionable. Participant observation blurs the border between researcher and participant, thus posing a number of dilemmas regarding free and informed consent.

It is true that participant observation challenges the biomedical hierarchical understanding of the relationships between researcher and research subjects. Not only can it be one and the same person, but also research participants are often more powerful and less vulnerable than the biomedical standard assumes them to be. In some cases, such as in critical policy research, participants are the source of risk to researchers.

Research Ethics Boards rarely reject proposals. They insists on revisions that are inconsistent with a given research methodology.  They force researchers to abandon their research plans altogether. Moreover, researchers often anticipate potential problems with ethics review and refrain from submitting a proposal based on research methodology that is not consistent with the biomedical standard, or make it conform to the expected norm.

Keep in mind that Research Ethics Boards manage risk by reviewing research protocols. But protocols are exactly the problem with participant observation research.  Participant observation is flexible and adaptive. It is a research method that is intended to be soft, reflexive, and interactive; when data collection and interpretation take place simultaneously and not sequentially. Knowledge can also originate in places where a detached observation would have never discovered it.

Igor Gontcharov

 


Once you have read this chapter, you should be able to:

  1. Understand what tertiary documents are, and the differences between primary, secondary and tertiary materials.
  2. Recognize the challenges that come from reading tertiary documents including scientific materials, field diaries, census data and statistical information.
  3. Find and locate science and science publication resources using databases.
  4. Perform the research steps required to engage in participant observation if your research project requires it.
  5. Locate, examine and extrapolate information from census data and other sources which use tables and statistics while making sure you don’t read more into the data than you should.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s