Communication Research Methods – Rebekah Bettner

Rebekah Bettner is a senior majoring in Communication Studies. She grew up in Richmond, Indiana and now resides in North Carolina.  This research paper, which was also presented via audio, was written for Professor Rosalie Aldrich’s CMCL-C424, Communication Research Methods, in Fall 2020.  Professor Aldrich comments that Rebekah’s “outline led to an excellent final presentation which critically analyzed a peer-reviewed journal article.” 

Critical Analysis of a Research Article

Communication Research Methods 

There is no doubt that technology has made our lives easier, and that it has replaced many of the simple tasks in our daily routines. You can pick up a phone and talk to your friend who is miles away. You can find potential romantic partners with just a click on an app. Mediated technology has all but eliminated the need for face-to-face interaction. This essay will critically analyze the reliability, validity, and value of an article in regards to mobile technology and its impact on college students’ ability to communicate. In this essay I will be breaking down and analyzing the study, methods used, and results of an article, then critically analyzing its strengths and weaknesses. Now we will begin to look at the article in question and the keys points of the article.

The article that I will be discussing in this essay is titled, “Smart Phones and Mediated Relationships: The Changing Face of Relational Communication,” and it was written by Loyd S. Pettegrew and Carolyn Day. It was published in The Review of Communication in 2015 (Pettegrew & Day, 2015, p122). The purpose of this study was to find out how mediated technology has altered interpersonal communication between college students and how it has influenced what communication means. Although the authors draw from interpersonal and mass communication theories throughout their study, this study is not driven by theory since it is exploratory in nature. They challenge how existing theories hold up in modern communication. A new theory is proposed for computer mediated communication that may explain why face-to-face communication may now have less meaning than mediated communication. This study will gather data from college students about their communication habits to be able to further examine this possibility. The sample of this study was a group of 526 communication and mass communication majors at a large university in the Southeast (Pettegrew & Day, 2015, p128). Representative random sampling was used to draw from 1,203 students who were in 2000 to 4000 level classes. Ethical guidelines were upheld because each student gave informed consent, the study was in no way harmful, and confidentiality was maintained.

The research design for this study was a survey. In order to conduct this study, the researchers created a 31 item questionnaire that consisted of interval, categorical, and nominal questions. Six quantitative research questions in particular were focused on in this study (Pettegrew & Day, 2015, p128). Students were randomly selected from classes in the communications department and told the intent of the study by the researchers. Each student was given a pencil and paper questionnaire that asked various quantitative and qualitative questions regarding their interpersonal communication experiences. When finished, each student was instructed to hand in their survey, whether they completed it or not, and all 526 were answered. The answers to interval and categorical questions were coded into Excel, while the qualitative answers were recorded verbatim and placed in broad category types.

We will now take a look at how the participants responded to the six quantitative research questions and what they indicate. The study results indicated three in four students believed that smartphones either greatly or moderately impacted how they communicate interpersonally. 50% of communication students said that their interpersonal communication was impacted greatly, and 29% said that it was only moderately impacted. About 15% said that mobile devices impacted communication a little and 7% said not at all (Pettegrew & Day, 2015, p129). The next set of data collected indicated that many participants spent several hours of their day with heavy use of mobile technology, keeping them in mediated contact with others. When asked about the important ways mediated technology helps develop relationships, the most common answers were, 26% said the ability to keep in touch, 17% that it was faster and easier, and 13% believed that it allowed them to be more comfortable and less awkward in conversation. Only 11% of students admitted that they had been without their phone for more than one day in the past month, but 32% said that they had only been without it for no more than 1-3 hours. 29% percent of students said they were not addicted to their phones, 36% said a little, 24% said a moderate amount, and 11% said a great deal. When asked about how mobile devices were better than face to face communication with romantic partners, nearly one third said that it wasn’t. Results indicated that the college students understand the impact of mobile devices on interpersonal communication. Mediated communication appears to have positive impacts, but also many negatives according to the responses provided. Now we will begin to examine the strengths and weaknesses of this study.

This study has many strengths. First, a pilot study was performed prior to handing out the questionnaire for the real deal. Two communication classes were involved in the pilot, and the study was revised to ensure comprehension and clarity. This adds to the reliability of the study. This study also has inter-rater reliability. Inter-rater reliability was tested because two separate people examined the results independently. They each coded the results then compared them using the inter-rater reliability test, which produced the .87 Siegel kappa. The results are not defective, but rather there is a major consensus that led to an acceptable inter-rater reliability. Another strength is that ethics were upheld. Researchers gained 100% approval by all involved after participants were told that the study was voluntary. Confidentiality was maintained and names were not put on any of the questionnaires, allowing the participants to remain anonymous. This study has face validity, at least based on the six questionnaire questions provided in the methods section. Researchers observed college students’ perspectives of mediated versus face-to-face communication and how they affect relationships. This means that the researchers should be asking questions that gather information that directly relate to how relationships are impacted by face to face or mediated communication. In terms of face validity, the measure makes sense. The questionnaire given to the college students asked six specific questions that make sense at face value and accomplished what the researchers are trying to figure out. As far as content validity goes, the researchers chose questions for their questionnaire that covered every aspect of what they were trying to find out. They did not have a complex concept they were researching, therefore they were able to easily cover all aspects of the topic in their research. Nothing important was left out of the questionnaire that would have benefited the research. There were a sufficient number of measures to conduct valid research. This study also utilized a strongly designed questionnaire. Questions were clear, only about the single issue at hand, and from the questions that were provided in the overview, none of them contained biased wording. Finally, the study employed the Likert Scale for the interval questions. This is a sound, tried and true method of measurement that enhances the study.

Although this study had a great deal of strengths, there are several weaknesses that need to be analyzed. The first weakness I would like to discuss is that there was no copy of the questionnaire included with the publication of this study. Although the study has face validity in terms of what was provided in the overview, we do not know the rest of the questions that were asked, therefore we cannot say that this study truly has face validity. Question order is another concern. Again, because we do not have a copy of the questions the college students were asked, we don’t know how they were ordered, and if the order of the questions could have played a role in how the students responded. This study also has limited generalizability. Because communication majors were specifically chosen, the only way to replicate this study would be to survey communication majors again. We can conclude that this study has no external validity. The next weakness we will discuss is how the researchers, who acknowledged that critics will accuse them of social desirability bias, specifically chose communication and mass communication students for this study. They claim that the bias of choosing communication majors was supposed to “have the effect of underestimating the degree to which mobile devices are influencing their interpersonal relationship development,” but it just seems as though they were trying to get specific results. The sample of college students was also admittedly convenient according to the authors. The sample was taken specifically from communication majors who are more likely to understand the importance of face-to-face communication, which could lead to skewed or biased results. Finally, since there was a researcher present for the survey, the Hawthorne effect is something to worry about. Participants may respond in ways that prevent their answers from tying them to their questionnaire, or they may fear someone watching them choose their answers. It is not uncommon for participants of research studies to change their answers when they know they are being observed.

From this study we can conclude that most college students understand the impact of mediated communication in their every-day lives. Interpersonal communication is impacted heavily due to mediated technology, and it deserves more research. The authors of this article were able to come to the conclusion that we can no longer solely rely on traditional interpersonal communication theories. The data they gathered indicates the need for further research and theory development for computer mediated communication, however this study did not provide enough evidence to begin theory development. This research topic is necessary to continue studying due to the increased mobile technology use in our culture. It is important to understand the effects of the devices we are using in our everyday lives.

References

Day, C., & Pettegrew, L. S. (2015) Smart phones and mediated relationships: The changing face

of mediated relationships. The Review of Communication, 15(2), 122-139. doi: 10.1080/15358593.2015.1044018