Second blog post - Theme 4: Quantitative research
The lecture gave a brief presentation of quantitative research. Basically quantitative research depend on numerical measurements. Data is collected by questionnaires where participants will be asked directly or with electronic tools. Data can also be collected by observation and the result will be interpreted by statistical tests. The distinction between quantitative and qualitative research have already been covered in the first blogpost of this theme. The differences and the pros and cons have already been stated also. And we did not get into it any further during the lecture.
The lecture also gave a brief presentation on how to perform a quantitative research. It all was pretty straight forward.
It concluded: Formulate a hypothesis, select a sample population to test it on and design the experiment. Collect the data and choose which analysis to perform. Match the analysis with the hypothesis and see if it corresponds to what was expected.
The questionnaire must be valid before usage. The questionnaire itself must have been tested by experiments. It is crucial that the questions are not angled or colored by the researchers.
In the paper by Bergström they presented a controlled experiment. In controlled experiment you manipulate an independent variable and follow up with an observation on how dependent variables vary as a result.
One thing that seems to be crucial in an experiment of the quantitative type is that all data is collected within the same framework. And that you treat all the participants the same as well as they all get tested with the same conditions. Otherwise you won’t get generalized data that can pass as valid.
I figure more reflections will be added after the seminars. One thing I came to think about was that if its common for researchers to “modify” tests to get the wanted result. In my previous class there was one person who had false data in his paper. And I was wondering the whole time how the hell he could collect 250 persons to run his tests. The computation and the result was misleading. In a research it seems to be necessary to have some kind of alibi to prove the validity of it.
I feel like you have a good understanding of quantitative research and you explained it really well. I thought your angle on weather or not it was common for researchers to "modify" their results were quite interesting, and something I was thinking about as well, as I've had some similar experiences. I remember in high school it wasn't very uncommon for people to make up results, instead of asking too many people, it was easier to simply ask some and then make some results up. This is of course very misleading, and I can't see it being ok, and the paper would (i would think) of course be discredited if anyone found out. But I still have to wonder how common it actually is, and if like you said, in a research it's necessary to have some kind of alibi to prove the validity of it.
SvaraRaderaThe question of whether or not it’s common that the results are being modified by the researcher has also crossed my mind and I think it’s a valid point to bring up to discussion. When the research are relying on, for example, a questionnaire where the participants answer anonymously, it’s not that difficult for the researcher to fill in a couple of answers on their own in order to modify it and get the wanted result. We can only hope that that is not the case and that researchers are honest but I agree with you that there should be some kind of alibi to prove the validity of it. However, in order to get your article published in a journal it needs to go through peer-to-peer review where they most likely control the validity, but I’m sure some cases slip through.
SvaraRaderaThanks for your reflection!
Hi! I also had this thought of how to make sure your research is valid and reliable. To make sure all results are accurate researcher takes lots of time to create an experiment framework and to test how it works. After-all, I believe there is no method that can assure that results are pure and honest. Maybe that's why scientists call their hypothesis significant, but never absolute truth?
SvaraRaderaThanks for your thoughts!
Interesting read!
SvaraRaderaEnsuring validity of results is hard indeed, and I believe it to be one of the greatest challenges of modern academia. With increasing amounts of technical tools to conduct research with, surveys and other quantitative (...and qualitative) methods are brought so far from the researcher that definitely ensuring the validity of the sample or the genuiness of their answers becomes impossible.
Perhaps one simply needs to respect and trust the academic honor codes that exist at many institutions around the world, and rely on the power of peer reviewing.
Hi! Thanks for your interesting reflection I enjoyed reading it. I also think that it is important to treat the participants in the same way to get data that can be compared with each other, especially in an experiment. However, I also think that in other methods for an example in a face-to-face interview it is also important to have that same attitude as social cues also influences the given answers. It might be in a different way then that of an experiment, but it is still important and affecting.
SvaraRaderaI agree with you in the terms of the validation of the results of the experiments - we've talked about it on one of the seminars. This term is strongly connected to the so called 'replicability crisis' when some experiments are almost impossible to reproduce in some areas and, therefore, impossible to check the outcome. At the same time, the percentage of the fake papers is still quite low according to the Wikipedia, probably, because academics care about their reputation.
SvaraRaderaAbout you coursemate: which kind of the survey did he conduct? If that was online, it's possible to get 250 responses, I see a lot of requests about the surveys in different Facebook groups and sometimes, if the topic seems interesting, I spend my time on answering it. Although, to be honest, I have huge doubts about the objectivity of the results: as mostly the youth (students) is targeted, and the older generations who would prefer paper format are omitted.
Thanks for the interesting opinions on it! I really like reading your reflection, which gives me a brief understanding of how the quantitative methods work on the research. I agreed that one thing is essential on conducting the experiment: all the data are collected in the same framework. You mentioned that it is crucial to treat all the participants the same situation otherwise you won't get the valid result.
SvaraRaderaHi,
SvaraRaderaGood summary of the theme. I was so surprised to read that someone in your previous class actually faked a study, but I can definitely imagine it being possible. I agree that there should be some sort of evidence to prove that the researcher has performed the study, but I think that should be proven by having the researcher explain the method process in detail in his essay. Also, answers and data from studies are supposed to be kept for a couple of years (don't remember exactly how many) in order for other researcher's to be able to access them and analyse them on their own. Considering your classmate got caught I suppose he messed up on one or both of these steps.
I think it is important to state that qualitative research does not only account for data collected through a questionnaire, but can also include e.g. controlled experiments and analyzing secondary data. I really like the part were you write about increasing validity treating participants in the same way during controlled experiments. Do you think that is possible? I think being aware of the influence one as a research has is the foundation, and using scripts could be one tool used.
SvaraRaderaYou also write about having some sort of alibi to prove the validity of research – I think that the data should be published along with the paper to ensure that other scholars can expand upon the data and criticize it. Not only opponents. This would be a way of bringing research into the sharing economy.
Allo,
SvaraRaderaYou are making a very valid point. In a research if we just focus on collecting, gathering data all over the place and that the base on what the research is made is false, misleading and can't be comprehensive, the information we gather will be totally wrong or at least not answering the initial research, the hypothesis.
And yes I think researchers that spend a lot of time in their research and can't find an answer to their problem after 10 years of their life would be most likely to cheat a bit in order to get a first answer in the good direction (At least ... thats what they do in the movies).
But I think when you you to show your study or if you want to publish it in a journal, then it will get verified and they will certainly see the mistakes you/ve made.
Manipulation of data is a topic I've already come across whilst commenting other blogs. Perhaps you had a discussion about it, but I agree that one should always be considering the intent behind research and this would be an important part of it. As another commenter said, I feel you could have expanded more on the topic and not just specifying a generic procedure.
SvaraRaderaIt is a bit surprising to hear that in a previous class, one student had false data in his paper. I wonder if there was any source of accountability required; perhaps a supervising professor to check in with before the research was presented? I believe most academic journals are peer-reviewed, so if you were to submit a paper for publication then it would be hopefully be determined that there were inconsistencies in the research.
SvaraRaderaThis made me think of another possible disadvantage of using anonymous online questionnaires. I suppose if the researcher really wanted to, they could log some of the questionnaires themselves in order to gain a bigger sample group or falsify data.