Today in class we all talked about the research method we researched and wrote about in our research journal last week. Dejan and I talked about the argumentative research method which I had found to be also called dialectic methods whilst he found it to be called argumentation theory however both dialectic method and argumentation theory have similar fundamental ideas of posing two ideas against each other to identify a truth and resolve the disagreement.
Here is the research methods that were covered in today’s class:
Secondary research – Making use of existing information. It is about going out there and finding all the articles you can to find about the subject to identify the main themes (e.g. 50% of the papers discovered x), it is trying to integrate a whole collection of others research and integrate a new idea.
It is the foundation of just about everything you do in a research context. Secondary research can be useful for anything, for example integrating new data sets to find new knowledge.
The strengths of secondary research are it is easy to access, and cheap with the internet
Weaknesses of secondary research are the information may not be exactly what you want, or relevant, and there is a lot of it.
Meta-analysis – It is secondary that is interested in quantitative data only, specifically statistical data sets. Papers that have in them statistical information in them.
It looks for exactly the same question that has been asked to more than one sample, it is widely used in pharmaceutical studies. It allows you to complete your own primary data on a smallish sample making the primary research affordable, whilst using the findings of other studies investigating the same thing, and treating them all as one study which gives a more credible result.
Evidence based medicine – this uses meta analysis to perform systematic reviews. A group of researchers (academic researchers and GPs) would get together and research the usefulness of a drug; they would get primary research from all around the world in English and other languages. Then they would analyze the research filtering out the incredible and invalid primary research papers, then once they have identified the valid and credible data they would then perform statistical analysis on those good research.
They would then make this available to doctors so they didn’t have to make a judgement on individual studies proving the performance of the drug.
An example in IT is computer assisted instruction
The benefits of this approach are high credibility/better overall knowledge/quality of the knowledge is good
The weakness of this approach are bias can still come through (can compound bias already present in the initial studies) and from the people who are doing the meta-analysis
Also it costs a lot in terms of money and time. There can be difficulty in finding of negative studies.
Randomized control trials (RCT) – This is effectively random drug trials. It uses blind (testers don’t know and double blind randomized trials testing one variable. – Used by big pharmaceutical uses it but also other kinds of medical interventions – Provide an unbiased study.
What it is trying to do is provide an unbiased study. The reason for wanting to reduce bias is because human bias can affect studies in many different ways you can’t imagine.
The person who knows which phial is a drug and which is a placebo is well up the ladder in the research because you don’t want to give away which is the drug and which is the placebo based on the doctors body language if they did know which was which.
The weaknesses of this approach are that they are very expensive and time consuming.
The strengths of this approach are they tend to give pretty clear results and remove most bias/ provide the best evidence that we can get
Case study research – Focuses on a case (people/group/company/event) that looks at one or more component of the case. The case can be a variety of things, e.g. a single company or collection of company.
e.g. collection of companies use of YouTube for marketing. You are trying to gain knowledge from a collection of cases for a specific variable or process.
Examples of case studies are:
- Exploratory (pilots) find questions/measurements to be used on a larger scale
- Illustrative/descriptive – To make the unfamiliar familiar by using common language (metaphor)
- Explanatory – cause/effect relationships, how/why things happen. e.g. why did TradeMe succeed and Weedle fail
- Cumulative – secondary case study – summaries on the case- integrating other information and refine them down to the single case you are studying
Example in IT – Widely used to explore to do a bigger study, to describe, greater insights into impact of IT
Case studies are useful in answering why, how questions, you wont come up with a definitive, final answer but you can learn a lot on the journey. You can look at the use of a technology on an existing company to see what changes were required by the company
Strengths of this approach are it is based on real life examples and so it is very practical and deals with real life issues. It provides in depth analysis
Bias – there is bias in you as researcher, and bias in the information. A technique to use to reduce bias is triangulation, it requires you to find data from 3 very different sources. If all 3 points agree then the data is more likely to be credible. If you have two agreeing sources and one strongly disagree. It gives you a somewhat objective viewpoints on the data.
Non generalisable (you can only draw common sense generalisations as case studies are specific to a particular situation e.g. apply a strategy to a different company it may not work in a new company which has different factors such as work culture) and hard to repeat. Loses lose its context (and therefore may not be useful).
You might do single instrumental case study were you learn about a single case such as a single company and then taking the insights about that company and you may be able to apply these insights to other companies. So although case studies are not generalisable you may be able to apply the insights to other companies.
Within a case study there may be:
Observational research – Observing behavior of something in its natural context (not just people).
It very much an initial approach when you’re not quite sure what your looking at. e.g. observing peoples behavior when they are faced with new software. You could look at where they click, etc. You can then say a good design for a website for people with eye defects are if they were being observed
Examples in IT are in the use of software, user experience research, interface between people.
Strengths of this approach are that it gives a close view of whats happening (with people vs interface). Additionally it gives a viewpoint to people who can;t give an opinion in any other way.
Weaknesses of this approach are it is subjective (based on the researchers interpretation), time-consuming, can be ethical concerns (covert particularly).
Types of observation – Naturalistic – Observing behaviors in a natural setting, but you have no input
Participant – Observing behaviors in a natural setting, but you have active participation.
Laboratory – Observing behaviors in a lab environment which can or cannot have researcher participation.
Observational research particularly when it is about people is open to the researchers interpretation.
Interviews – Conversation between two people to extract specific information. The interview can be structured (with set questions), or it can be a free flowing conversation (unstructured), or semi-structured interview.
Can be used as a follow up to other research e.g. after a survey or as a prelude to a survey
You could use a structured interview if you know what kinds of answers you want, and/or know what questions you want to ask. Possibly to encourage deeper/critical thinking. Structured interviews allows for comparison after the interviews of multiple people is complete, it doesn’t give such a rich picture of what one person thinks but it allows you to say two out of the ten people can
If you want to get credibility for a thought or compare the answers then a structured interview is adventurous.
But if you want to investigate something in depth then unstructured or semi structured interviews are best; e.g. PHD interview which starts with a structured questions to compare the students response to other students responses. Then it was followed by unstructured interview where responses that the student says are picked up by the interviewer and a follow up question is asked for clarification and/or expansion.
Differences between direct and indirect questions. Silence is an important tool.
Examples in IT where you are trying to find out a persons opinions, also can be used to investigate in a richer sense a persons knowledge of the topic. By interviewing people in different job roles you are getting viewpoints on a software systems implementation for example in an information rich way.
Strengths of this approach are it is flexible, useful to get good detail and clarification.
Weaknesses of this approach are the use of leading questions from the interviewer can influence the response of the interviewee. Bias/expensive/time consuming/may not interview the right people/ not always truthful/people will tell you what they think you want to know.
Next week Dejan and I will be talking about the argumentative method.