Post theme 6

This week we learned about qualitative research and case study research. I began with selecting and reading a paper that used a qualitative method. After that I read the paper Eisenhardt where she described what case study research is and how it can be done. My learnings from Eisenhardt’s paper was then used to analyze another paper that used the case study method. Unfortunately I didn’t have the opportunity to discuss the given literature and my chosen papers with my friends before writing the blog post that was written before this theme. So I didn’t have a chance to hear my friends’ ideas and thoughts about this week’s theme. After going to the seminar I fortunately got the opportunity to listen to other students thoughts about this weeks theme, which gave me a bigger understanding.

I didn’t have much knowledge about case study prior to the theme so what I primarily learned during this week was what case study research is and how it can be conducted.  After reading the papers about case study I was a bit confused about what case study is because that case study can be many things, e.g. it can include both qualitative and quantitative methods but also just either of them, it can include several cases or just one. This broad description made it hard to find what the true core of what case study is.  But as usual the seminar gave me a bigger understanding. Leif Dahlberg described it as an event or object that has a “natural frame”, for example an organization. He also compared it with how doctors look at patients or police officers looks at a crime scene, a comparison I will take with me since it describes case study research well. What I thought was particularly difficult with case study research was to differentiate it with other research methods. But at the seminar I learned that in case study research the case is the subject for the research, not a hypothesis or theory. The case study should instead generate new hypothesis or theories that then can be tested with other research methods.

Of course I learned some about qualitative research but this was an area I was much more familiar with. What I did learn though was how the coding of the results found in the qualitative methods can be done, for example by using a software or by letting different researches interpret the results found. 

Post theme 5

Unfortunately I couldn’t attend this week’s lecture so I didn’t get to chance to hear Eva-Lotta Sallnäs Pysander when she elaborated about her research and how she used a collaborative setting to the findings in her paper.

I prepared this weeks theme by reading the given literature and by reflecting the questions following each paper. What I also did this week was to discuss the papers with two friends who is also following this course. We have done this prior to every theme but this time I thought that the discussion was worth mentioning. The discussion with my friends gave me further understanding and different perspectives on the given papers, so I’m glad that we did it.

What I learned this week in particular was the method to use collaborative setting and the benefits of doing so. Earlier I hadn’t really thought of this method. This benefits with this method was that you are in some sort doing a “think aloud”-analysis (Wikipedia 2014) without forcing your participants into doing so. In my bachelor-essay earlier this year and from earlier KTH courses I have done some think aloud-analyses but I have always thought that it was a unnatural setting, the participants seems uncomfortable when telling what their thinking. In the collaborative setting you will still obtain the thoughts of your participants, but in a much more natural way since the participants are discussing with each other.

References

Wikipedia. (2014). Think Aloud-protocol. Access: http://en.wikipedia.org/wiki/Think_aloud_protocol [2014-10-10]

Pre theme 6

Qualitative methods

I read the paper “The psychological functions of avatars and alt(s): A qualitative study” by Gilbert et al. (2014) that was published in the journal Computers in Human Behavior who has a impact factor of 2.273. The paper is about multiple personas in Second Life and how those are functioning for the user.

Which qualitative method or methods are used in the paper? Which are the benefits and limitations of using these methods?

The qualitative method used in this paper is semi-structured interviews with open-ended questions with 24 participants. The interviews were conducted in a virtual Second Life environment, so the researchers and the participants never met face-to-face. The benefits with semi-structured interviews are that the open-ended questions invite the participants to speak more freely. This means that you will get a deeper understanding of your subject of study, since the participants can tell you the connections between different phenomena. But this also means that every interview can be time-consuming, so you cannot have a big amount of participants, which is why results from qualitative studies rarely can be generalized. The researches also have to interpret the transcripts from the interviews, which means that the results will be subjective. In this study the interviews were conducted in the environment that was the subject of the research, the interviews about Second Life were held in a Second Life environment. So the researchers got a contextual understanding of the subject.

What did you learn about qualitative methods from reading the paper?

What I learned when reading this paper is how you could code the findings from the qualitative method, and this coding where something that I weren’t familiar with before. The coding development in this study was made by first reviewing the transcripts of four interviews iteratively, with the purpose of trying to find different categories well connected to the hypothesis so they could categorize the statements made by the participants. I really liked this approach and I’m happy that I learned about it since this is a fairly easy method to sum and get a understanding of what was actually said during the interviews. The coding also introduces a element of quantitative method since you can with numbers and percentages say how many said this or that, and lacking of numbers is often a big limitation the qualitative research.

Which are the main methodological problems of the study? How could the use of the qualitative method or methods have been improved?

The main methodological problem with this study was the fact that the interviews where made in a virtual Second Life environment. Even though I said under the first question that gives a contextual understanding this introduces a couple a implications for this study in particular. The results of this study said that people make the avatars and different personas so they could act as a different person than themselves; they often distance their physical self from their avatars. This could mean that the participants weren’t truly themselves during the semi-structured interviews either, so the researchers got the answers given by the avatars not the honest answers and opinions from the physical person behind the avatars.

Case study

Briefly explain to a first year university student what a case study is.

A case study is when you are trying to draw conclusion by studying one or several “real-life” examples, and it often gives you a deeper understanding of a phenomena. The method used if often a mix a quantitative and qualitative methods, but it can be just one of them. By using and studying cases this will provide description, you can test theory or produce new theory. (Eisenhardt, 1989)

                     

Use the "Process of Building Theory from Case Study Research" (Eisenhardt, summarized in Table 1) to analyze the strengths and weaknesses of your selected paper.

I read the paper “A case study of Augmented Reality simulation system application in a chemistry course” by Cai, Wang, & Chiang (2014)that was published in the journal Computers in Human Behavior. The case study is testing the theory that augmented reality gives students in junior high school a better understanding of chemistry. I think that the case study in this paper is conducted pretty well according to the process-model given by Eisenhardt (1989). They begin with defining their research questions and they select a case connected to them. Then they develop a augmented reality tool which they test with several different methods, like questionnaires, field studies and interviews. So these first steps from Eisenhardt’s model are conducted and are the studies strengths. But after that the papers weaknesses appears, they don’t do any cross-case pattern search, they don’t review their results iteratively to confirm their theory and to not involve and literature to compare their findings with others. Maybe there are some elements of these steps in the paper, but if so they haven’t described them well since I can’t tell. But I’m asking myself if these last steps have to be done thoroughly when you are only trying to test a theory and not generate new, which was the case in this paper.

References

Eisenhardt, K. M. (1989). Building Theories from Case Study Research. Academy of Management Review, 14(4), 532-550.

Cai, S., Wang, X., & Chiang, F.-K. (2014). A case study of Augmented Reality simulation system application in a chemistry course. Computers in Human Behavior, 37(0), 31–40. doi:http://dx.doi.org/10.1016/j.chb.2014.04.018

Gilbert, R., Thadani, V., Handy, C., Andrews, H., Sguigna, T., Sasso, A., & Payne, S. (2014). The psychological functions of avatars and alt(s): A qualitative study. Computers in Human Behavior, 32(0), 1–8. doi:http://dx.doi.org/10.1016/j.chb.2013.11.007

Pre theme 5

Haibo Li et. al

How can media technologies be evaluated?

In the paper by Li and colleagues they tested their prototype with a usability test. Many media technologies are developed with the purpose of people using them, so that’s why a usability test is suitable. In this paper they used the ISO definition of usability; Effectiveness, Efficiency and Satisfaction when designing their usability test, a very common way of designing usability tests.

What role will prototypes play in research?

Prototypes can be developed if your hypothesis requires one, i.e. your hypothesis is about some device or service that don’t exist yet. This can be the case in media technology research. Of course you can develop a prototype if the existing similar ones are expensive, and your budget don’t allow buying the existing ones. The prototype is tested in order to try to answer your hypothesis.

Why could it be necessary to develop a proof of concept prototype?

A proof of concept prototype is a very early stage of a finished product. This kind of prototype is necessary since you can evaluate advantages and disadvantages with your idea, and you will get the answers early in your research process. You can get answers to questions like; is people interested in your idea? Is your idea usable? Without having to waste too much time to develop a more finished product.

What are characteristics and limitations of prototypes?

The main characteristic with a prototype is that it’s not the finished product, it’s a more simplified version of it. The prototype is developed to test your hypothesis or concept, not to be released on the market after you have conducted your research. This means that some flaws are accepted on a prototype, it don’t have to look or function like the end-product, in Haibo Li’s paper the vibrating phone had a PCB board connected to it. This is also one limitation with prototypes; these flaws can affect the participants in the usability test, and if so this can affect the result.

Prototypes also have the limitation that the researchers can get emotionally attached to it, the researches really want the prototype to succeed and in that case over valuating it, which can effect the analyze of the results and the designing of the research. To prevent that you shouldn’t develop any advanced prototypes, for example with functions that aren’t needed to answer your hypothesis.

How can design research be communicated/presented?

If you used a prototype in your research it should be well presented and in some cases also the process when developing it. This includes images, mockups, diagrams, graphs, tables and text. Especially the text connected to the prototype should handle things like which goals you had with the prototype and which hurdles you stumbled upon during the development. Otherwise a design research should include similar things; images, mockups, diagrams, graphs, tables and text. I think the most important thing is to be transparent, do not leave anything unmentioned if that could affect the research, even the smallest events can be interesting for your readers.

Sallnäs et. al

How does a collaborative setting differ from a single user setting as regards methodology used and the results obtained?

The collaborative setting invites the participants to talk about what their thoughts to each other naturally, you can say that you in some sort are doing a “think aloud”-analysis but without forcing the participants into doing it. The communication between the sighted and the non-sighted participant lets the researcher know what in particular the non-sighted person needs in order to complete the task. These needs can then easily be compared between the audio and non-audio pairs in the results. So the most interesting result obtained with this method is what the non-sighted participants lacks to complete the tasks, and by comparing these between the audio and non-audio pairs, you can easily find what positive aspects the audio has when complementing virtual haptic environment.

How can qualitative and quantitative methods in the same study complement each other?

They can complement each other by in the qualitative method you can get answers to why certain results were obtained in the quantitative method. The qualitative method gives you more understanding and depth to the raw data obtained in the quantitative method, for example you can find connections between the quantitative results.

How can using both subjective and objective methods give a better understanding of a phenomenon?
I think this is connected to the quantitative and qualitative methods. As I said above the qualitative method (the subjective method) gives you more understanding and depth about the result acquired in the quantitative method (the objective method).

Post theme 4

This week we learned about scientific research and especially about the advantages and disadvantages of quantitative and qualitative research. I started this week’s theme by reading the paper from Olle Bälter and colleagues. The paper illustrated very well and I learned a lot about the complexity of conducting quantitative research with a big number of respondents.

For example there is always a issue of drop-outs, but during the workshop Olle taught us how we can decrease this number. He purposed several steps including a invitation (without the survey attached) and several reminders (both with phone, snail-mail and e-mail). It all origins from that you should treat your respondents professionally and with respect. Bälters paper also described the issue of outer parameters that can affect the result, especially when it is a quantitative research over time. This was also a problem in the other paper I selected to read and I learned a lot about the issue.

During the workshop, which was a bit messy due to the large amount of students and the small room it was conducted in, we talked about the advantages and disadvantages of quantitative and qualitative research as well as paper-based and web-bases questionnaires. The things that were brought up weren’t really new to me. One thing that I haven’t thought about earlier, and that I learned during the workshop, was the usage of interactive web-questionnaires and it’s advantages. Bälter presented a interactive questionnaire that he had done that was about public health. This questionnaire provided things like BMI and suggestions to the respondent on how he/she can improve his/hers health while the respondent answered the questions. So since the respondent got valuable feedback from answering the question, they got an incentive to do the questionnaire and answer the questions truthfully and thoroughly.