Class notes: Additional work I have done on the class exercises

Today we had Liz instead of Clare for taking our class because Clare was unwell. I did further work on on the class exercise to define 10 questions I want to answer about my project subject and then I identified 3 good questions from that list of 10 and then I identified the fundamental concept I want to explore.

From there I choose one of the questions and I defined what research methods I would use to answer the question, and a relevant title.

Identify a potential project

Construct 10 (interesting!) questions about your topic

  1. What automated testing tools are available for free, preferably open source?
  2. What are the perceived benefits and downsides of these automated testing tools according to other developers?
  3. What is the perception towards the adoption of automated testing tools from the Koha community?
  4. What automated testing tools are the easiest for non-developers to learn and use? as some of the Koha testers are librarians who do not know how to code
  5. Would the adoption of automated testing tools speed up the Koha bug workflow?
  6. What is the most suitable automated testing tool for Koha?
  7. Are there automated testing tools that test if patches meet the project’s coding guidelines, i.e. not just testing functionality?
  8. Which of the tools use a record/playback or manual coded test case development?
  9. Does the automated testing tool use a model-based testing?

 

Identify three good questions/ideas. For each of these 3 suggest what you could do to answer/explore it.

  1. What automated testing tools are available for free, preferably open source? Secondary research to identify the range of free automated testing tools available
  2. What is the most suitable automated testing tool for Koha? Primary research using exploratory research specifically a comparative study. I will write/create test cases for a variety of automated testing tools to test a variety of Koha patches and will compare how each performs to identify the best performing tool.
  3. Would the adoption of automated testing tools speed up the Koha bug workflow? The answer for this question leads on from question 2 because each patch that I test using a automated testing tool will have the time it takes to test recorded and this will be compared against the time it takes to manually follow a test plan.

 

What is the fundamental goal underlying each of the 3 questions?

To find a suitable free automated testing tool to use in the Koha project that is both easy to use (so it’s use isn’t restricted to developer testers) and speeds up the Koha bug workflow.

  1. Making a choice
  • Choose one of the questions and make a more detailed description of what you would do to answer/explore the question i.e. a first draft of your list of activities

What is the most suitable free automated testing tool for Koha? This will be answered through performing secondary research from academic journals, industry whitepapers, and online courses to identify the available free automated testing tools.

Followed by exploratory research method (which is a primary research method) to write/create a test case for each of the identified tools to be run on 3 different patches from the Koha bugtracker (known as Koha BugZilla). The time and ease to write each tools test case and then run the automated test will be recorded. Then each of the 3 patches will be manually tested by me to give a time and ease comparative value for manual testing.

The fastest and easiest to use automated testing tool will be able to be identified from the comparative study. Therefore I will have identified the most suitable tool for use in the Koha project. I will promote this tool to the Koha community through the #koha irc channel, and mailing list.

Also this suitable tool will be compared against the time/ease to performing each of the three patches manually and so I will be able to identify if the use of an automated testing tool for patch testing (regression testing) would be useful for the Koha project

  • Write a title for this possible project.

Identification and justification for the use of the most suitable automated testing tool for the Koha Integrated Library Management System (ILMS) project.

Identifying a project exercises: Class notes

Today in class we talked about assignment 3. I went and had a chat with Clare about my project proposal and how it can be integrated into assignment 3. I learned that I should hopefully get feedback to my project proposal by the end of this week.

Clare recommended for the rest of the class to go through and define what they found interesting in IT, we did this through a warmup exercise, answering these sentences:

The IT area/subject I have most enjoyed is learning about systems analysis and design, in particular the new development methodologies that can be utilized instead of SSADM and OOSAD.

The IT area/subject I least enjoyed is networking as I find it very frustrating.

The IT area/subject I was most interested in is systems analysis and design methodology.

The one IT thing I never want to do again is developing 2D game in the Unity game engine. (stay away)

I choose to study IT because I can help develop tools to be used by millions of people around the world and make a real difference. 

If I couldn’t study IT I would study zoology or archaeology because I have always had a fascination for animals and history.

When I was a kid I wanted to be zoologist and was planning to study a Bachelor of Science at Canterbury University for this purpose.

One IT thing I would like to learn more about is machine learning and neural networks in Artificial Intelligence (A.I.).

We did this exercise to identify what we are interested in that can be integrated into our project making it more interesting and engaging.

Then we went through an exercise taking the subject we identified that we were interested in during the warmup exercise and then expanded about it:

  1.  Broad Research/Project Interest Area

Description of area of IT that interests you

Automated testing tools automate functionality and regression testing. Regression testing is particularly useful for established software development projects where developers are submitting patches which alter the existing behaviour or fix an existing bug, but it can also be used for functionality testing for feature enhancements.

Why is it interesting to you

For the Koha product that I work on there are more contributing developers than testers, what this means is the whole bug workflow is slowed down with developers writing patches and no-one signing them off, at present there are 158 patches requiring signoffs.

The longer a patch is left waiting for testing and signoff the more likely it is to become out of date as the master branch has changed so much and so when a tester finally tries testing the patch it will not apply successfully because the project has moved on so much.

Automated testing tools would reduce the bug list requiring signoffs by automating one third of the process (the functionality testing), whilst the code review, and running the qa test over the code would still have to be performed by the tester.

Three things you know about it

  • Automated testing tools are mainly used for regression and functionality testing, meaning it can automate functionality testing plans but it cannot check if patch meets coding guidelines
  • Many automated testing tools can run on test plans by people that do not have coding experience, the test plans that the tools follow are based on simple english.
  • I know how to write functional test plans for human testers
  • ‘Robot’ is a recommended automated testing tool because it allows you to not only perform functionality testing but it can also compare screenshots of U.I.’s to check design changes.

Three things you believe about it

  • Selenium isn’t stable and it often breaks during a regression test
  • Automated testing tools are hard to maintain

Three things you don’t know about it

  • What automated testing tool is most appropriate to the Koha product
  • How the automated testing tool knows what the previous behaviour before a patch was applied was, i.e. do you have to run it firstly over the master branch and then run it following through a simple english test plan with the patch applied, or if you have to write the current behaviour in the test plan in addition to the new behaviour
  • How much faster automated testing tools are in the Koha project by the time a simple english test plan has been written than having a tester following a test plan manually

Would you rather: Do something, Research it or Do both

I would like to do research and do something. Specifically I would like to investigate what automated testing tools are available and what their pros and cons are, and then I would like to perform primary research on the identified automated testing tools by seeing how effective and reliable they are in automating the testing according to the test plans

Find an online resource about it – (post url)

https://www.qatestingtools.com/compare-bdd-testing-tools

http://searchsoftwarequality.techtarget.com/feature/A-look-at-todays-top-automated-functional-testing-tools

http://searchsoftwarequality.techtarget.com/tip/Automation-testing-Seven-tips-for-functional-test-design

Identify a potential project

Construct 10 (interesting!) questions about your topic

  1. What automated testing tools are available for free, preferably open source?
  2. What are the perceived benefits and downsides to these free automated testing tools according to other developers?
  3. What is the perception towards the adoption of automated testing tools from the Koha community?
  4. What automated testing tools are the easiest for non-developers to learn and use? as some of the Koha testers are librarians who do not know how to code
  5. Would the adoption of automated testing tools speed up the Koha bug workflow?
  6. What is the most suitable automated testing tool for Koha?
  7. Are there automated testing tools that test if patches meet the project’s coding guidelines, i.e. not just testing functionality?

 

Academic paper analysis tips

How to recognize a academic paper.

Here are some tips that Clare gave us:

  • If an article is peer reviewed then it is highly likely to state this
  • Most journal articles are peer reviewed
  • Academic journals usually have a affiliation and the contact details for the authors for the readers to be able to contact them regarding there research
  • Academic papers have an abstract – a summary of the report which sums up the whole of the academic paper.
  • If a paper doesn’t have a abstract or references then it is highly likely not to be an academic paper

 

How to find academic papers?

Google Scholar – allows you to filter search results that you would find in Google.

  • You can access a free PDF of each article on Google Scholar on a link to the right of the title of each article on Google Scholar
  • You can save each article by clicking ‘Save’
  • You can generate a APA citation by clicking the ‘cite’ link under the article

NMIT databases – There are none specifically about IT

  • ProQuest is business related and Information Systems and Information Technology can be considered under this banner concept
  • Science direct
  • ERIC

Steps to skim through an academic paper to determine if the paper is well written, credible and valid:

  1. Pay attention to the title – The title should be between 7-10 words it should be clear, but concise.
  2. Pay attention to the author – Find out who wrote/performed the research
  3. Pay attention to the abstract – What is the broad area of what the research is about, what the authors did, and what the papers conclusions are.
  4. Read the introduction – This should set out where it has going. Emphasis on where we are going.
  5. Read the conclusion – Set out what the paper was about, it should not contain any new information. Should cover where it has been.
  6. Check if the research ontology, epistemology, and research method are suitable: This will allow you to determine if the research is credible and valid.

 

Assignment 2

Read through one of these two academic papers and write a critical review of the paper.

 

This weeks blog post is:

  • Choose one of our two papers
  • Read the paper
  • Answer the following questions:
      1. Did the abstract tell you the three things I said it should? If not, what did it tell you? (NB If your paper doesn’t have an abstract, it is not an academic research paper!!! Go and find another one!)
      2. What seems to be the research question(s) they were trying to answer ?
      3. What  method(s) did they use to answer the question(s)
      4. How credible do you think the paper is? (hint: look at who authors are and where and when it is published also compare what they were asking with what they did)
      5. Did you agree, or not, with what they wrote in their conclusion? Why?
      6. Briefly describe two things that you learnt from the paper.

    In no more than 250 of your own words (i.e. a paraphrase), describe what the paper is about – you could start with “This paper describes……….”

 

 

Project 2017

I have clarified the number of hours I will need to do for my project I will be doing between 300-350 hours doing the development work for Catalyst and I will be writing up my project report, keeping my blog up to date over the course of 100 hours.

Clare thought it would be nice if I wrote up a project report and if there was a report from a member of my team at work, because I will setting a precedent of a remote work placement.

Research Approaches part 2 and what is credible research?

Today we went through the remainder of the research methods which I will cover in this research journal entry, then we started discussing what sources we can consider credible sources for research

 

Research methods

Experimental research:

Proving or disproving a hypothesis over a series of tests on various groups, tests consist of manipulation /controlling of variables in a controlled environment. This method is part of the scientific paradigm.

It is testing an idea (this is idea is a hypothesis which is testable in a controlled and repeatable way) which is a concept from the scientific paradigm.

Example: Measuring effects of soft drinks/measuring the effects of mobile devices on eyesight over a period of time.

Strengths of this approach are: Repeatable, generalisable, easier to see if it is valid

Weaknesses of this approach is: Control can be difficult, confounding variables, statistics can still be interpreted in a biased way.

 

Social scientific paradigm research methods:

Exploratory research:

Qualitative research that lays the foundation for further work – getting to understand/know the subject/focuses on ideas on how to do research in the area -Preliminary work

When you find a subject has not been researched before, or if there are a few papers that exist but you are not happy with how they approached the research then you investigate the subject in a very broad way and then you can find a potential for future research.

e.g. Investigation of ambiguous questions e.g. deconstructing a question like the ‘best laptop’ question . Or investigating a brand new technology (technology).

Another example is exploring peoples perception on the dangers of true A.I. whether they see it as scary or exciting, this would be a beginning of much further work into that area

Strengths of this approach: Very helpful to investigate new areas, provide road-maps for where we might go, or what might be useful research. Can clarify what the biases are which can make eventual results more reliable.

Even no results are useful as they can help to identify further useful work.

Weaknesses of this approach: Less status, expensive (time and money), not conclusive and shouldn’t be considered as such. Always needs further clarification can be biased.

 

 

Discourse analysis

Analysis of spoken or written words to discover the meaning behind them.

For example: Different words are used for the same concept in an organization. So the CEO might refer to students as customer whilst the teachers might refer to students as students. The different words used to identify students convey different meanings.

This research method is analyzing the interpretation of language.

Discourse analysis can be used in courts as judges have to interpret the wording of laws.

Strengths of this approach: Its a familiar strategy that we all use informally (we all know how to do it), very powerful in extracting ‘real’ meaning.

Disadvantages of this approach: Very hard to do with non-expert language users, very hard to do, and time consuming (personal bias).

 

Action research

Has to be flexible, ongoing, process. You must keep an eye on the process and change when needed – researcher is part of the research (instead of trying to distance themselves from the research to try to improve the objectivity of the research).

Taking an agile focused approach to your research by investigating something and changing it and then testing it again. Also everyone is involved in the research including the researcher. So it is almost the iterative, user centric spiral model of Agile methods.

This research method can resolve some problem which traditional way of research can not do – and/or create something new before something fails, works well alongside the people who will use the end result of your research.

 

Focus groups

Getting a small, diverse group together asking questions on a specific topic. Guided and open discussion – The researcher guides the discussion at the start and then hopes the conversation opens up as members of the group chime in their opinion and then the focus group takes it from there.

The researcher is active at the start of the focus group meeting, and as the focus group goes on they become less active. They don’t just want a reply to one question they want others to add to the answer and make for a richer research method than an interview.

It is a form of exploratory research because you can analyze what the members say, and discover new areas to investigate.

Used for market research (business area) to test out peoples reactions of products and services.

e.g. Feedback on an app (in development), Sandra research – focus groups of high school students talking about IT careers.

Strengths of this approach: People are more open in a group discussion than in a one on one interview, and they feed off what others say

Weaknesses of this approach: What one person dominates the discussion, and influence others towards ‘group think’.

 

Design science research

Where you are creating something; e.g. new app, new OS and its a way of actually looking at a useful artifact, testing it, and evaluating it.

Used mainly in constructing, IT and education.

Practical in nature in that you are actually doing something. The intention of this research is to help  people.

 

Argumentation research

Posing two different viewpoints and supporting them with logical and emotional evidence – one is a thesis and the other point of view (pov) is the antithesis. (Social science paradigm)

Also known as the argumentation theory.

e.g. Two designers disagree over the design of an app – after comparing they came to a synthesis of the two povs.

Helps to resolve disagreements, allows the questioning/challenging of accepted wisdom.

Argument is very dependent on how well someone can present it, biased, synthesis may not be reached

 

 

Credible research

Sources for research are (with credible sources highlighted):

Library

  • Journals – Probably out of date
  • Newspapers/magazine
  • Textbooks – Tend to be out of date, and accepted wisdom rather than anything new, exciting or challenging
  • General
  • Previous projects
  • Library database – ProQuest, Eric –  Anything you find through library databases are usually peer reviewed, and so likely to be the most credible source in the Library.

Internet

  • White papers – Research at IBM puts out  good information. It is specifically funded by a company or government
  • Google scholar – Search engine where you find evidence. Alternatives are Research-gate, academic.edu.  Very good idea as it filters out non academic journals and academic books.
  • Journals – Slightly higher value than printed journals as they will likely be more up to date.
  • newspaper/magazines
  • e-books
  • websites
  • magazines
  • videos
  • online tutorials
  • images, graphics etc
  • code repository
  • databases of information
  • data sets – For example the census results, which are very highly credible
  • public records
  • Technical papers – Very high credibility. Usually not particularly biased, instead it is a description/discussion of a technical subject like new OS.
  • self publishing
  • vanity publishing
  • MOOCs – Massive Open Online Course – University courses made available to the public by universities such as Harvard. High credibility.

Other things

  • Other people
  • Your own experience
  • TV
  • experts – You have to be careful to determine if they are expert in the area they talking about
  • classrooms
  • data sets
  • feedback
  • observation
  • your own research

 

Its not just knowing the source of research is important, we need to know other things. For example if research came from a blog, some blogs are very credible, whilst others will not be at all credible. So you have to look at the credibility of the author in that case. So it depends

 

 

Research approaches

Today in class we all talked about the research method we researched and wrote about in our research journal last week. Dejan and I talked about the argumentative research method which I had found to be also called dialectic methods whilst he found it to be called argumentation theory however both dialectic method and argumentation theory have similar fundamental ideas of posing two ideas against each other to identify a truth and resolve the disagreement.

Here is the research methods that were covered in today’s class:

Secondary research – Making use of existing information. It is about going out there and finding all the articles you can to find about the subject to identify the main themes (e.g. 50% of the papers discovered x), it is trying to integrate a whole collection of others research and integrate a new idea.

It is the foundation of just about everything you do in a research context. Secondary research can be useful for anything, for example integrating new data sets to find new knowledge.

The strengths of secondary research are it is easy to access, and cheap with the internet

Weaknesses of secondary research are the information may not be exactly what you want, or relevant, and there is a lot of it.

Meta-analysis – It is secondary that is interested in quantitative data only, specifically statistical data sets. Papers that have in them statistical information in them.

It looks for exactly the same question that has been asked to more than one sample, it is widely used in pharmaceutical studies. It allows you to complete your own primary data on a smallish sample making the primary research affordable, whilst using the findings of other studies investigating the same thing, and treating them all as one study which gives a more credible result.

Evidence based medicine – this uses meta analysis to perform systematic reviews. A group of researchers (academic researchers and GPs) would get together and research the usefulness of a drug; they would get primary research from all around the world in English and other languages. Then they would analyze the research filtering out the incredible and invalid primary research papers, then once they have identified the valid and credible data they would then perform statistical analysis on those good research.

They would then make this available to doctors so they didn’t have to make a judgement on individual studies proving the performance of the drug.

An example in IT is computer assisted instruction

The benefits of this approach are high credibility/better overall knowledge/quality of the knowledge is good

The weakness of this approach are bias can still come through (can compound bias already present in the initial studies) and from the people who are doing the meta-analysis

Also it costs a lot in terms of money and time.  There can be difficulty in finding of negative studies.

Randomized control trials (RCT) – This is effectively random drug trials. It uses blind (testers don’t know and double blind randomized trials testing one variable. – Used by big pharmaceutical uses it but also other kinds of medical interventions – Provide an unbiased study.

What it is trying to do is provide an unbiased study. The reason for wanting to reduce bias is because human bias can affect studies in many different ways you can’t imagine.

The person who knows which phial is a drug and which is a placebo is well up the ladder in the research because you don’t want to give away which is the drug and which is the placebo based on the doctors body language if they did know which was which.

The weaknesses of this approach are that they are very expensive and time consuming.

The strengths of this approach are they tend to give pretty clear results and remove most bias/ provide the best evidence that we can get

Case study research – Focuses on a case (people/group/company/event) that looks at one or more component of the case. The case can be a variety of things, e.g. a single company or collection of company.

e.g. collection of companies use of YouTube for marketing. You are trying to gain knowledge from a collection of cases for a specific variable or process.

Examples of case studies are:

  • Exploratory (pilots) find questions/measurements to be used on a larger scale
  • Illustrative/descriptive – To make the unfamiliar familiar by using common language (metaphor)
  • Explanatory – cause/effect relationships, how/why things happen. e.g. why did TradeMe succeed and Weedle fail
  • Cumulative – secondary case study – summaries on the case- integrating other information and refine them down to the single case you are studying

Example in IT – Widely used to explore to do a bigger study, to describe, greater insights into impact of IT

Case studies are useful in answering why, how questions, you wont come up with a definitive, final answer but you can learn a lot on the journey. You can look at the use of a technology on an existing company to see what changes were required by the company

Strengths of this approach are it is based on real life examples and so it is very practical and deals with real life issues. It provides in depth analysis

Bias – there is bias in you as researcher, and bias in the information. A technique to use to reduce bias is triangulation, it requires you to find data from 3 very different sources. If all 3 points agree then the data is more likely to be credible. If you have two agreeing sources and one strongly disagree. It gives you a somewhat objective viewpoints on the data.

Non generalisable (you can only draw common sense generalisations as case studies are specific to a particular situation e.g. apply a strategy to a different company it may not work in a new company which has different factors such as work culture) and hard to repeat. Loses lose its context (and therefore may not be useful).

You might do single instrumental case study were you learn about a single case such as a single company and then taking the  insights about that company and you may be able to apply these insights to other companies. So although case studies are not generalisable you may be able to apply the insights to other companies.

Within a case study there may be:

Observational research – Observing behavior of something in its natural context (not just people).

It very much an initial approach when you’re not quite sure what your looking at. e.g. observing peoples behavior when they are faced with new software. You could look at where they click, etc. You can then say a good design for a website for people with eye defects are if they were being observed

Examples in IT are in the use of software, user experience research, interface between people.

Strengths of this approach are that it gives a close view of whats happening (with people vs interface). Additionally it gives a viewpoint to people who can;t give an opinion in any other way.

Weaknesses of this approach are it is subjective (based on the researchers interpretation), time-consuming, can be ethical concerns (covert particularly).

Types of observation – Naturalistic – Observing behaviors in a natural setting, but you have no input

Participant – Observing behaviors in a natural setting, but you have active participation.

Laboratory – Observing behaviors in a lab environment which can or cannot have researcher participation.

Observational research particularly when it is about people is open to the researchers interpretation.

Interviews – Conversation between two people to extract specific information. The interview can be structured (with set questions), or it can be a free flowing conversation (unstructured), or semi-structured interview.

Can be used as a follow up to other research e.g. after a survey or as a prelude to a survey

You could use a structured interview if you know what kinds of answers you want, and/or know what questions you want to ask. Possibly to encourage deeper/critical thinking. Structured interviews allows for comparison after the interviews of multiple people is complete, it doesn’t give such a rich picture of what one person thinks but it allows you to say two out of the ten people can

If you want to get credibility for a thought or compare the answers then a structured interview is adventurous.

But if you want to investigate something in depth then unstructured or semi structured interviews are best; e.g. PHD interview which starts with a structured questions to compare the students response to other students responses. Then it was followed  by unstructured interview where responses that the student says are picked up by the interviewer and a follow up question is asked for clarification and/or expansion.

Differences between direct and indirect questions. Silence is an important tool.

Examples in IT where you are trying to find out a persons opinions, also can be used to investigate in a richer sense a persons knowledge of the topic. By interviewing people in different job roles you are getting viewpoints on a software systems implementation for example in an information rich way.

Strengths of this approach are it is flexible, useful to get good detail and clarification.

Weaknesses of this approach are the use of leading questions from the interviewer can influence the response of the interviewee. Bias/expensive/time consuming/may not interview the right people/ not always truthful/people will tell you what they think you want to know.

Next week Dejan and I will be talking about the argumentative method.

What is credibility and validity?

How do we decide if research is valid and credible? Well we have 4 things to look out for:

  1. Does the epistemology match the ontology?

If a research question clearly requires an answer using the scientific paradigm are the methods used to answer the question a match for the scientific paradigm. e.g. you don’t want to do surveys to ascertain peoples opinions.

If the research paradigm and methodologies match then then the research is more likely to be valid and credible.

Here’s an example:

What is the impact of fun in work?

  • You could try to determine the question is looking at the effect of fun at work on productivity and so use the scientific paradigm. Through measurement, and observational methods this is a scientific ontological viewpoint.
  • Whilst wanting peoples opinions would mean you would want a qualitative methodology.

Something to keep in mind when determining if the research question and methods match is to consider what information the question is capable of giving you, and what approach do you want to take to find out the information. It depends on how you interpret the question, so this means the research question you write needs to be sufficiently specific clear to make it easy to determine the methodology to perform to answer the question.

2. Was the method followed sufficiently rigorously? Is the method that the researcher used rigorous enough.

This look at the method the author used to determine if they have they done it well. Because we are just starting in learning about the research methodologies so it would be hard for us to determine from looking at a academic paper if the methodology was followed rigorously enough.

However if you can identify that the research methodology was not followed rigorously then you can view the research is not credible.

3. Who did the work/research? Do they seem credible?

This is pretty straightforward with internet sources to find out more about the author, although good academic journals state who the author is and if they are credible. For example is author known to be expert in this field.

Also consider if the article states if/who funded the research. Just because research is funded by an corporation does not mean it is not credible, its just important for research to state if the researcher is funded as it makes that bias public. It is important to remember if no funder listed you can’t assume its not funded.

Also if a author states their biases at the start of the paper they are making a positive move in this direction. e.g. if a student wrote an article about student engagement in IT then they should declare the bias that they are a student and so this is made public.

4. Where and when was the work made public (or published)

Has article been published in peer review (which requires at least 2,3 or more people have reviewed the paper) in peer reviewed journal. Peer review works by having the authors name and location hidden whilst the review is taking place.  The reviewers check how results match with what others have found, quality of the work. Only if all reviewers agree to publish the paper then the authors name is shown.

Peer review is more likely to be credible, but not necessarily valid.

Also consider when was the research published, this is pertinent in fast-changing fields like IT.  Older research is some areas of IT like security are likely t be less relevant if they are older than around 2 years.

Reminder: Communications of the ACM is a credible IT publication.

We did a class exercise to get us thinking about if questions were good or not, and what knowledge and methodology we should use to answer the question. Here is what I came up with along with what the class discussion determined:

  1. Which of these two laptops gives the best performance?

Is this a good question?

No because it does not say the performance that you are comparing them on. For example it is not stating if the question is comparing the CPU performance, i.e. which has most efficient CPU. Without stating what you are comparing on this question is not specific enough.

It does not state performance in what, e.g. for gaming.

Class thoughts:  The use of the word ‘best performance’ does not specify the context. Someone should define ‘best performance’

What kind of knowledge/evidence would be needed to answer it?

You would need to use a scientific paradigm, in other words this question will require a quantitative approach to answering it because it is comparing engineering which has numerically measurable values. E.g. CPU performance can be measured this is a objective truth rather than having to ask someone else’s opinion.

Class thought: Performance of each component

Numeric data

If you interpret the question on UX of OS as performance then the question morphs

How would I gain/gather that knowledge/evidence?

You would use scientific instruments and monitoring software to measure quantitative values on the performance on the laptops. These two quantitative values can then be compared in the research analysis.

Class thoughts:
Benchmarking

Hardware comparison

Experimentation

Mixed method – If People were important to question as well as quantitative data to answer the question. If the question was not fully defined.  

  1. Are virtual worlds like Second Life or Minecraft useful for teaching?

Is this a good question?

No because as with the last question this question is not specific enough, it doesn’t state what subject/if any subject/concept you are using these software products for to teach and comparing on. i.e. I believe this question would be more advantageous if you asked ‘Are virtual worlds like SecondLife or Minecraft useful for teaching physics?’

Because you would be able to more easily narrow down what is being  compared making the methodology easier and more effective to implement.

Class thoughts:

Worse than the first question, because it is not specific enough because it uses the vague word “useful” what do you define as useful

Additionally you do not define what the virtual worlds you are testing and who they are useful for

What kind of knowledge/evidence would be needed to answer it?

I believe you would need 3 groups, 1) No software, 2) Second life 3) Minecraft

This would require both quantitative and qualitative data to answer it because you would need qualitative data from the course tutor such as student engagement which is an observable occasion.

You would also need quantitative data such as course grades to show the grades that the students got so you can see if the (however this could be considered qualitative depending on the subject e.g. maths is more objective than English)

Class notes:
Subjective – opinions and perceptions

Could be numeric e.g. pass rates

 

How would I gain/gather that knowledge/evidence?

Survey of tutors in the 3 groups to gauge student engagement.

Collection of the student grades to gather the students qualitative data.

Class notes:

Comparison between taught topics in and out world (some numeric and some words)

Mixed methods

When talking about people in this context who are you talking about.

  1. Why don’t many school students (16-18yrs old) choose to study IT at Polytechnic or University?

Is this a good question?

This is a bad question because it uses a subjective measurement, i.e. it is saying not many school students choose to study IT, but it is subjective what not many is

Class notes:
Bad question because:

  • What’s defined as ‘many’
  • No context of which students
  • It is being specific about certain things (should be school leavers)
  • No geographic context
  • Assumes!! That the statement is true. It should state if not many students choose to study IT, spell out this assumption g. the percentage increase of students studying IT has increased by 5% only, why is this?

Good- Being specific about certain things

What kind of knowledge/evidence would be needed to answer it?

You would need to ascertain qualitative data because this is a social sciences research, i.e. you are asking the students their opinion on the IT courses at polytech and uni and so you want to retrieve word data and subjective opinions based on their biases.

Class notes:
Subjective – Its asking why. Words/interpretations

How would I gain/gather that knowledge/evidence?

You would use surveys to gain the fundamental overview of a large group of students then you would use interviews to get more in depth qualitative data.

Class notes:
Survey (qualitative or quantitative)

Interviews

Focus group – Group of people interested in something and discuss a subject and see what is said

  1. Which ISP in NZ gives the best value for money?

Is this a good question?

This is a good question because it states the location, and the measurement variable i.e. what is the best value.

This has the whole scope of the research outlined in the question because it was more specific than the previous questions.

Class thoughts:
What does best value mean? Not specific enough

Different geographic regions even in NZ
Asking for some specific answer but could be several

What kind of knowledge/evidence would be needed to answer it?

You would need quantitative data to be able to compare the best value because the ISP’s provide a measurable service to the customer i.e. you can measure the number of GB they use and the amount of money they spend.

Class thoughts:

Could be either numbers (cost/benefit) or words (customer service)

How would I gain/gather that knowledge/evidence?

You would need to get the Gb/$ ratio so you could compare the different ISP prices vs the data you get. T do this you may need to get this quantitatively information from the publicly released information form the companies which is in the public domain.

Class thoughts:

Analysis of the plans

Survey of current customers

  1. How do I feel about trying to work with slow internet speeds?

Is this a good question?

This is not a good question as again it is too general, it does not define what the person is trying to study (i.e. if they are studying carpentry they may not need to use the internet vs someone doing a web development course and is using the internet all day will need the internet.

Class thoughts:

Because it uses ‘I’ it is using the creative paradigm questions

Subject

What kind of knowledge/evidence would be needed to answer it?

This is very much a qualitative elicitation of the views of people. Because it is inquiring as to the audiences personal views based on their biases.

Class thoughts:

Emotional reaction

How would I gain/gather that knowledge/evidence?

You would perform a survey to get the opinions of a wide group of the audience and then use interviews to get the opinion of randomly chosen individuals in more depth to ask further open-ended question s that they are unlikely to bother answering in a survey.

Class thoughts:

Self reflection as you asking the person what they think

Being interviewed

Poem/meme

  1. What are the main security issues associated with ‘cloud computing’

Is this a good question?

This is a good question because it is outlining the scope of the research it is stating you are looking at the security issues and the area you are investigating is cloud computing.

However, I object to the word ‘main’ because this is not defined meaning you don’t know what the main security issues what part of cloud computing are you looking at the security issues of

Class thoughts:

Whose main are we talking about, i.e. what is main

Extremely broad – As cloud computing is a huge field

What is meant by cloud computing

What kind of knowledge/evidence would be needed to answer it?

You would want to look at quantitative data to look at the faults in the cloud computing, in other words you will want to compare the quantitative data on highest security failures to determine the main security issues.

Class thoughts:

Words/perceptions

Numbers – lost business/cost of downtime

How would I gain/gather that knowledge/evidence?

Using quantitative data from system logs to see failures.

Class thoughts:

Surveys and interview

What is the takeaway message of this class?

When looking at others research to determine if it is valid and credible then you need to ask these three questions:

  1. Is it a good question
  2. What knowledge are we looking for?  (e.g. emotion, facts)
  3. What methods were used? (and were they followed correctly)

Something that really stood out for me about today’s class was how a question which is not specific enough or it makes assumptions can have a huge bearing on if people think your research is credible. So to practice writing good research questions I am going to keep the needs for specificity and no assumptions in mind for all future IT interest research questions for this research journal.

In preparation for ext week I need to do the following:

Research the research method of argumentative. I will write a research journal entry about argumentative research methods by answering these questions:

  1. What is it? (short description of how it works) Link to good resource as well
  2. What kinds of questions or problems might it be useful for?
  3. How could it be used in IT research? (think of an example)
  4. What are the strengths of the approach?
  5. What are the weaknesses?

How do ontology, epistemology and methodologies fit together? What is post-modernism and fuzzy logic? Class notes

Today in class we talked about the concepts of ontology, epistemology, and methodology and how they interact within paradigms. So effectively the class was following up on what we researched since last lesson clarifying it .

We also tried to define what post-modernism is (which is certainly hard to do), as well as what fuzzy logic is.

Ontology

It asks the question: What exists?  What is real? Ontology basically looks at our beliefs of what exists.

This relates to what we said last week. The majority of people will believe something is real but others won’t and they will say what we see is in our imagination e.g. Most of us believe the roof of old St John is red, but some people might not agree that the roof it red.

We all exist on a continuum regarding our ontological position. There are two extremes realism and nominalism with the middle ground being constructivism.

20170315_120518.jpg

More detail about the ontological positions:

Realism – Objective reality: If someone was 100% realist then they would think a $20 bill is just a bit of green plasticised paper. Something is real, it exists but they don’t consider the value that society has placed on it.

The more scientific your background the m0re likely you are to have a realist ontological position. The realism ontological position generally concentrates on testing and proving/disproving theories.

Also it is important to note it is way more convenient to be a realist as it is a more socially accepted view.

Nominalism – Subjective reality: This ontological position believes nothing exists unless we as an individual imagine it.  If we were 100% nominalist we would believe that the only things that exist are things that we as individuals constructed.

The more artsy you are the more likely you are to be in the nominalism end of the ontological continuum.

Constructivism – This is the middle ground between the extremes of realism and nominalism. Constructivism believes things that we believe as existing are just constructs of human society. For example the government which is just a societal construct for a group of people that govern us, or money which is an object or concept that has been given a value by society.

This is seeing socially constructed reality. e.g. what we see around us has been constructed by us socially as a group. This group could be huge, human race, or small, a family who believes something exists.

Money is a good example of a socially constructed reality, because society has decided it has a value.

If we are dealing with something that is socially constructed then we as a society can change something. e.g. can devalue our currency. The government sets the socially constructed reality. Here we are trying to discover things, and build theories.

I was thinking that another example of how society can change societal reality is through free speech and the freedom to protest this is how the societal reality of slavery was stopped in 19th century Britain for example.

In research we need to realize that we move along the ontological position continuum

Epistemology

It asks how do we know?

In other words epistemology is the theory of knowledge. In terms of research it is how do we know things.

Again as with ontology epistemic positions exist along a continuum but our epistemic position is related to our ontological position.

Epistemological positions

Positivism: This is the belief that things exist (so it links in with the realist ontological position) and it believes there are methods we can use to prove things exist. The methodology we use with realism and positivism are a quantitative approach using numbers, measurement, observation. i.e. the use of statistics.

Interpretivism: This epistemic position links in with the constructivism ontological position and it believes that to understand social constructs you have to to interpret what people are saying, and doing. The problem with this is that your own bias is coming into play.

The methodology to perform research in the world of constructivism and interpretivism is  qualitative (the data you are collecting is primarily words, e.g. survey (surveys ask the same question in multiple ways to check if you are answering correctly and have understood the question), observations (observing peoples use of technology to observe if they are having issues that may not have been elicited by using a survey), focus groups, analyze literature, case studies). All the time we’re doing this we try to bring in the objectivism.

Antipositivism: This epistemic position links in with the nominalism ontological position (which again believes things can only exist subjectively for example if you have a pain in your knee it can’t be measured but you hope the person your saying you have a pain in your knee to has had a similar experience and so can relate and empathize). It is the opposite of positivism.

The methodology to perform research in the world of nominalism and antipositivism is  through creativity:  e.g. art, books, poems, fine art, pictures, music.   It is trying to share subjective things.

Research paradigms

A paradigm is a way of thinking about the world, based on ontological, epistemological and methodological assumptions.

We combine the relevant ontological and epistemic positions (I mentioned which ontological and epistemic positions were linked together above)

There are three main research paradigms:

Scientific paradigm –  Also can be called modernism. It is a paradigm that says things do have a reality, and it is possible to prove it. This is a very Western perspective, which has been spread across the world through Western technology. It is based on the realist ontological position and positivist epistemic position.

Social scientific paradigm  – This is systematic research, because you follow a plan of research which is not necessarily repeatable because it is dealing with people. However the bigger the sample size then the more likely the results are to be repeatable, because statistically speaking a larger sample size produces a more repeatable mean results. It is based on the constructivist ontological position and interpretist epistemic position.

 Creative paradigm – This is the creative or emotive research, trying to prove a subjective reality. We won’t be concentrating much on this research paradigm in this class. It is based on the nominalist ontological position and antipositivism epistemic position.

The problem when trying to apply one of these research paradigms to IT research such as when we are building an IT artifact (e.g. OS, application, programming language) is it doesn’t really fit under any particular research paradigm, because it requires both qualitative and quantitative research. So a new research paradigm was conceived:

Design science research paradigm – It is a way of looking at and evaluating something that has been built. It identifies the problem ->  design a solution -> build a solution ->  evaluates the solution.

e.g. problem – How to teach database to beginners

Solution New solution to teach db to students

Build solution : Build teaching resources

Evaluate it – Evaluate the teaching answers

This paradigm is used in IT, engineering and education.

This research paradigm uses a combination of quantitative, qualitative research methodologies

Post-modernism: What is it?

As soon as you try to define something you are not being post modernist any more. Post-modernism comes back to quantum physics, the revelation light can be wave and particle at the same time.

To anyone coming from the realist ontological position to say something can be two things when you look at it, cuts away at their ontological position and its associated quantitative methodology.

Post modernism is the questioning of the certainty of modernism and the acceptance everyone can be right, because the world does not exist in binary.

It started in late 19th century, which was when the scientific paradigm (modernism) was at its peak.

It questions and challenges the norm, such as challenging the gender binary of male and female. Another way to challenge modernism (scientific paradigm) is through constructivism (social scientific paradigm) as it acknowledges what are accepted truths are based on society believes.

Fuzzy logic

It allows computers to think of shades of grey, not black and white. It will have a huge affect in A.I. as we want A.I. to think along perspectives (along a continuum) which is how humans think. Only then will we have something truly intelligent.

Fuzzy logic allows computers to make the decisions like humans based on non-quantitative data. It allows computers to make decisions based on words. It took a long time to be accepted in the West as a lot of computer science believed in binaries (modernism) but now it is being used in Western technological breakthroughs.

Fuzzy set – Knowledge in real world

Fuzzy rules – Fuzzy logic rules

An example of fuzzy rule application is how driver-less cars know how to continuously brake, applying different amount of break to apply at a specific time.

Why is all this theory useful for RES701?

What you want to design/build will require a different research paradigm. i.e. if you are going to build a new system then you’ll want to use the design science research. So you’ll probably won’t do a lot of the context (analyzing literature) you’ll do more designing the solution, building it, and testing it.

When analyzing a source you should identify the ontological position the author was in, then check if the epistemology and methodology match the ontological position. If they don’t match then it is unlikely to be a credible source.

i.e. You have to identify what it is they want to know, and then what did they do to find that out and if they don’t match and so it is unlikely to be credible.

Unless the writer of the academic literature states the limitations of their research and what their biases are.

For or research it is a good idea to state what our bias was as this makes it easy for the reader as they don’t have to try to elicit your bias from your writings.

Take away message of the class was: There are different research paradigms we should use based on what we want to know, so basically its similar to SYD701 where we are learning that some systems development methodologies are better suited to the development of some systems than others.

What is true and truth? Class notes

Today we got into groups (I was in a group with Jonathan, Becca and Jared) and we discussed some very in-depth questions about what we think truth and reality is.

The point of this exercise (in addition to getting us thinking about what ‘true’ is) was to see how everyone has different answers to these intellectually challenging questions, in some groups peoples answers were totally different to those of their team mates.

In my groups case we quickly worked through the answers writing our individual ideas into a Google Doc. We all seemed to have fairly consistent opinions on the answers to the questions and so there was not a lot of verbal debate.

Whereas David and Harry’s group had a lot of discussion and debate when answering every question.

But why did the members of David group all have different answers?

Well each persons answer to the question ‘what is true?’ is based on their personal bias. A personal bias is basically our worldview based on our experiences. Because my group had very few disagreements in the writing of our answers (which I tried testing occasionally by asking the others about what they thought of the answers written for some of the questions) it could be hypothesized that we all have relatively similar personal biases in relation to the topic of truth.

Here are the answers my group agreed on:

  1.  Is there a  difference between ‘knowing’ something and ‘having knowledge’ of something? Explain

Knowing something is to have a light or little understanding of the thing or simply the knowledge of its existence. I.E you heard something from one source

Having knowledge is having an in depth understanding of the thing I.E you heard something from one source but then backed up that from several other sources.

 

  1. What is ‘truth’?

“The quality or state of being true”

“That which is true or in accordance with fact or reality.”

“A fact or belief that is accepted as true.”

The most popularly believed and backed up belief of the populace

 

  1. What do we really mean when we say something is ‘true’?

It’s our belief and bias that we believe said thing is true. Whereas it may not be proven to be true.

 

  1. Is there a difference between knowing something is ‘true’ and believing that something is ‘true’?

Can you know something is true? If you can, knowing something is true means that it is true, whereas believing something is true is more a matter of a personal feeling or belief and may not be true to someone else.

 

  1. What is the difference between subjective and objective ‘truth’?

Subjective truth is something that is believed by person based on how they see the world through their personal bias.

Objective truth is not determined by a personal bias but proven by independent, observable or unobservable facts.

 

  1. What is a ‘fact’ and  can ‘facts’ change ?

A fact is a piece of information that proves a theory, or truth. Facts can change as technology and scientific inquiry proves a truth incorrect. Many related facts can back up a theory, however science like evolution changes as one theory is proven to fit the facts more closely than another so the current theory is proven untrue.

 

  1. How do we discover if something is ‘true’ or not ?

To a point we can determine that something is true based on which answer has the most raw data backing it up. However this does not necessarily ensure the that answer is 100% correct. Only that is it is true at the time.

Its should also be noted that this true can often be skewed by what the general populace believes and religious, cultural, etc influence

 

  1. “We do not see things as they are but as we are” Anais Nin. What does this mean?

We view the world through a worldview based on our upbringing and personal biases, this means that everything that we see is based on ourselves rather than the thing itself.

 

  1.  “Reality is an illusion, albeit a very persistent one”  Einstein. What does this mean?

“Life is a presentation of choices. Wherever you are now exactly represents the sum of your previous decisions, actions and in-actions. At any given time the number of choices available to us is limited not by our capabilities but by our assumptions and awareness. The greater our awareness the greater the possibilities our lives can entertain. A limited awareness puts limits on the possibilities and futures we could hope to achieve”

 

I can’t remember who said it, but one philosopher said that reality is 2D and that we exist on a painting. Every decision has already been decided for us as it is already painted onto the campus.

Stephen Hawking said, that he believes the universe we live in has a predetermined destination and path, but that the path and destination is unknown to us, so we may as well live our lives as if the universe was not pre-determined.

  1. Is there a difference between ‘true’ and ‘valid’? Explain!

When something is true it is an absolute, whereas when something is valid it has the possibility of being true but may not be. I.E if you look up a phone number in the phone book then it is a true number, however if you just type numbers in the phone, the number is (probably) valid but may not have a number attached to it.

 

Class discussion

After answering the questions as groups, we had a class discussion to investigate the different viewpoints of the different groups, and here’s what we came up with:

What is the difference between knowledge and having knowledge?

Knowledge is tacit knowledge. It is information that you have due to having experienced, or made observations. This information is in your mind rather than being external to you (written down) (“What is tacit knowledge? definition and meaning – BusinessDictionary.com,” 2017).

For example you can drive a car because you have had the experience of learning to drive, after driving for long enough you can get to the unconscious competence stage of the four stages of competence (discussed below), in other words you can drive without even having to consciously think about what to do.

Whereas having knowledge is an example of explicit knowledge. Explicit knowledge is “Articulated knowledge, expressed and recorded as words, numbers, codes, mathematical and scientific formulae, and musical notations” (“What is explicit knowledge? definition and meaning – BusinessDictionary.com,” 2017). I understand it to be knowledge that is outside of your mind; it is written down or stored externally. e.g. A car manual describing how the car pedals work.

A relevant book that Clare recommended was: To have and to be – Erich Fromm

What is truth?

Truth is often a emotional concept rather than an intellectual concept. In other words we often have a different emotional feeling reacting to something we think is true as opposed to something we think is untrue.

We correspond truth with morality – if something is true it is good.

When we as individuals say something is true it is often based on our beliefs and experiences. Our belief is based on our biases and life experiences.

Something is more likely to actually be valid if it is both observed and believed by a large group of people. I would say this is because there is a reduced personal bias due to a greater sample group, meaning if they concur then an objective truth is more likely to have been discovered.

An example is if you put enough car-denying people in front of cars traveling at speed towards them, and they get flattened then not only are they likely to revise their opinion of cars existence (if they survive) but also they would have observed that cars definitely do exist because they have been flattened by one, and so the objective truth of cars existence has been discovered by a large sample group believing and observing it.

 

What is a fact?

A fact can be defined differently based on the field it is in.

In scientific research (which is both systematic and has repeatable results) facts are the results of the scientific research and this research can be repeated many times to give the same results meaning that the results are less likely to be based on human bias but instead be objective truths. Of course the interpretation of the scientific facts can be influenced by human bias.

Whereas in social sciences research, like history, it is very subjective and based on the bias of the history writers and the society in which they live. This is because history is written by the winners.

Whilst alternative facts are different interpretations of the same thing.

How do we discover if something is ‘true’ or not?

As I discussed above for scientific inquiry it is relatively easy to prove if something is true or not. Whereas in social sciences it is harder to validate.

4 stages of Competence: This is the belief that whenever we learn anything we go through the following stages as we slowly understand the concept:

  1. Unconscious incompetence (we don’t know what we don’t know)
  2. Conscious incompetence ( we have enough understanding to realize that we don’t understand the subject)
  3. Conscious competence (we understand the subject but have to think exactly what we’re doing when applying what we know)
  4. Unconscious competence (we can do something without having to think).

4 stages.jpg

(Csabai, n.d.)

Relating this to the concept of truth I believe that steps 3 and 4 fall heavily under the concept of subjective truth because we are basing our understanding on the knowledge we have internalized through our personal bias.

In other words I see our personal bias as the window that we see the world through.

As we learn a subject, for example I am trying to learn python at the moment, even though the syntax of the language will be the same in everyone’s understanding (i.e. it is an objective truth that is observed and believed by many people) the way I  understand, internalize and memorize it is based on my ‘window’.

I can hook a piece of syntax onto something that is significant to me e.g. some Python syntax might be similar to another language I already know and so this is a way for me to memorize the syntax which is unique to me. Meaning I am in stage 2-3 and I am gaining a subjective truth of the subject which is tested whenever I run my code, if my code fails then after troubleshooting the error my subjective truth is revised.

This was a fascinating topic and now I am going to investigate what ontology and epistemology are in the next journal entry.

Bibliography:

What is tacit knowledge? definition and meaning – BusinessDictionary.com. (2017). Retrieved March 8, 2017, from http://www.businessdictionary.com/definition/tacit-knowledge.html

Csabai, M. (n.d.). 4 Stages of Learning any new skill. Retrieved March 8, 2017, from http://mindinmotion.co.za/4-stages-of-learning-anything/

 

Formal vs Informal research personal reflection

Something that really stood out to me in today’s class that I didn’t realize was just how similar the research we do everyday is to formal research performed both by IT professionals (when designing and developing new systems and applications) and academics.

What do I mean by everyday research. Well an example from my life recently is I recently researched which tour company to use in a trip to Egypt and Jordan I am planning to visit at the end of this year (hence the header image of the pyramids).

The process I went through to select a suitable tour to pay to go on was:

  1. Identify that I wanted to go to Egypt and that I wanted to go on a tour
  2. General search for tour companies available in Egypt and their tours
  3. Filtered the available tours based on the dates I was going to be in Egypt and the budget I had
  4. Compared the tours based on where in Egypt they visited and their price
  5. Simplified down what I wanted out of a tour to Egypt and Jordan based on what I had read in tour summaries and reviews
  6. Made decision to go with the ‘Highlights of Egypt’ tour by G Adventures.
  7. Went to the travel agent and booked my selected tour
  8. Justified the choice to myself, and told friends and family about it

To transform the research process of everyday (informal) research (as shown above) into formal research there are 3 further constraints that need to be applied:

  • Critical study – Looking at all sides of the argument to make sure the whole spectrum is covered and then to reflect on all the sources and draw your own conclusions, justifying them.
  • Scientific inquiry method – Performing research in a systematic way that gives a repeatable result. This often works best for the hard sciences such as chemistry where the same reagents mixed together will always give the same result.
  • Systematic – This is where you follow a particular plan to perform research. When the systematic approach is applied to social sciences it will often not give repeatable results because unlike hard sciences social sciences studies people who are fickle.

I used to think formal research would have been far more differentiated than the process I went through due to those performing the formal research often being higher level tertiary students completing PHD theses.

Having said this I had realized that it is not just academics that perform formal research its IT professionals as well because in my summer IT internship (as discussed in ‘My initial thoughts on research‘ post) where I had to develop a new feature for Koha library management system I had to research similar features in competing systems, as well as solutions to coding errors. So research skills are necessary throughout the whole of the systems development life cycle not just the analysis and design phase.