• Users Online: 129
  • Print this page
  • Email this page

Table of Contents
Year : 2021  |  Volume : 8  |  Issue : 4  |  Page : 237-242

Development and psychometric properties of the online health information-seeking skill scale

1 Department of Medical-surgical Nursing, Trauma Nursing Research Center, Kashan, Iran
2 Department of Medical-surgical Nursing, School of Nursing and Midwifery, Kashan University of Medical Sciences, Kashan, Iran
3 Department of Midwifery, School of Nursing and Midwifery, Kashan University of Medical Sciences, Kashan, Iran

Date of Submission20-Jul-2021
Date of Decision06-Sep-2021
Date of Acceptance30-Nov-2021
Date of Web Publication30-Dec-2021

Correspondence Address:
Mr. Jalal Hashemi-Demne
School of Nursing, Kashan University of Medical Sciences, Kashan, Isfahan Province
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/iahs.iahs_148_21

Rights and Permissions

Aims: Given the dearth of research about middle-aged people's ability to seek online health information in developing countries, this study was conducted to develop the Online Health Information-Seeking Skill Scale (OHI-SSS) and evaluate its psychometric properties. Materials and Methods: This is a methodological study in which a scale was developed within three factors and was validated by face validity, content validity, and structural validity methods. Four hundred and twenty middle-aged individuals completed the questionnaires. Internal consistency and test–retest were used to evaluate the reliability of the scale. Finding: The initial scale, consisting of 38 items on a 5-point Likert scale, was reduced to a 26-item scale following face and content validity measurement. The exploratory factor analysis extracted three subscales in OHIO-SSS that includes “information reception,” “provide and exchange information, and “identification and trust.” Internal consistency of the scale was confirmed by Cronbach's alpha coefficient (0.93). Cronbach's alpha coefficient for subscales was 0.87, 0.795, and 0.74, respectively. Reliability analysis with test–retest revealed an acceptable estimate for the total score (intraclass correlation coefficient = 0.92). Conclusion: The 20-item OHI-SSS has acceptable validity and reliability. Therefore, it can be employed as an appropriate instrument for the evaluation of middle-aged people's skills in seeking online health information.

Keywords: Adults, health, information seeking, Internet, skill

How to cite this article:
Izadi-Avanji FS, Hashemi-Demne J, Kafaei-Atrian M. Development and psychometric properties of the online health information-seeking skill scale. Int Arch Health Sci 2021;8:237-42

How to cite this URL:
Izadi-Avanji FS, Hashemi-Demne J, Kafaei-Atrian M. Development and psychometric properties of the online health information-seeking skill scale. Int Arch Health Sci [serial online] 2021 [cited 2023 Jan 31];8:237-42. Available from: http://www.iahs.kaums.ac.ir/text.asp?2021/8/4/237/334279

  Introduction Top

The number of health-related websites has increased to be much more than information sites.[1] Many people seek online health information to get answers to their questions.[2] Online health information covers topics such as the exchange of experiences and finding of support, as well as information and advice.[3] It was shown in a study in Kuwait (2018) that 93.2% of people used the Internet and 62.9% of them sought online health information to promote self-care behaviors.[4]

Self-care is the most important form of primary care.[5] Providing online health information is a new and emerging phenomenon and one of the sub-branches of telecare.[6] This method of care creates an opportunity for interaction between users and health-care professionals,[7] provided that users have the skills of accessing online information. Paying attention to individuals' skills in using online health information, the distinctive nature of users, psychological and social issues, privacy, information quality, and legality are new topics that need to be explored.[2]

Holman and Lorig (2013) state that resource-utilizing skills are one of the core skills of self-care.[8] Many health-care providers introduce resources to participants but do not teach them how to use these resources.[5] Therefore, before introducing resources, especially electronic resources, it is necessary to evaluate the skill of using them in the user by a valid tool.

Literature review shows that the available instruments are not developed specifically to assess people's skills in seeking online health information. The instruments have been developed by reviewing the articles and have not used the direct experiences of individuals.[1],[9],[10],[11],[12],[13]

For example, Razmak and Bélanger developed a tool to assess the use of certain technologies by physicians and patients.[12] Wong and Cheung conducted a study to investigate the relationship between online health information seeking and eHealth literacy among patients attending a primary care clinic.[13] Kelly et al. developed an instrument for measuring the potential consequences and experiences a person may encounter when using health-related websites.[1]

Addressing the needs of the middle-aged population is critical to online health information seeking. The reason is that this group of people enters the aging stage after middle age and their health and medical needs impose a heavy burden on the health-care system.[14] The skill of seeking online health information will aid greatly in meeting the ever-increasing needs of this rapidly growing population for cost-effective health care. For this reason, health policymakers and even web developers need to have basic information about the skill of middle-aged people for online health information seeking. Owing to the low number of studies in this area, and heterogeneous measurement of skills in seeking online health information, this study was conducted to develop a scale of measuring middle-aged people's skills for online health information seeking.

  Materials and Methods Top

This methodological study was conducted between February 2020 and April 2021 on middle-aged people referred to Urban Comprehensive Health Service Centers in Kashan. The methodological studies include two qualitative and quantitative phases, respectively.[15]

In the first phase of the study, the item pool was developed through interviews with 16 middle-aged people and a literature review. Interviews were conducted with middle-aged people who had access to the Internet via smartphones, computers, or laptops. The main interview questions included: For what purposes do you use the Internet? What are your uses of the Internet with health and hygiene? Interviews were analyzed using the content analysis method, and Graneheim and Lundman's approach. Finally, 86 codes and 3 main themes (identification and trust, information exchange, and provision and information reception) were extracted from the interviews. Categories and subcategories were used to generate the primary item pool for the Online Health Information-Seeking Skill Scale (OHI-SSS). The available literature and instruments will also be used for item generation.

In the second phase of the study, the initial scale was developed with 36 items. The scoring tool was a 5-point Likert scale with the following options: “very good = 5,” “good = 4,” “somewhat = 3,” “little = 2,” and “never = 1.” The higher the scores acquired by the participants, the higher would be the skill of seeking online health information.

After the development of the initial scale, the psychometric properties including face validity (quantitative and qualitative), content validity (quantitative and qualitative), structural validity, and reliability were examined, respectively.

To evaluate the qualitative and quantitative face validity, the scale was provided to 12 middle-aged people to determine the level of difficulty of the phrases, the relationship of the phrases with the main purpose of the scale, ambiguity, and inadequacies in the meaning of words.[16] In addition, they were asked to rate the importance of each item on a 5-point Likert scale ranging from 1 (not important at all) to 5 (very important).

Then, the impact score of each item was calculated according to the following formula. If the item impact was equal to or greater than 1.5, the item was retained.[17] Formula: Item impact = the importance × frequency (percent). At this stage, two items had a score of <1.5 and, therefore, were deleted.

To test the qualitative and quantitative content validity, eight nursing professionals who were experts in the development of scale and six health information management and technology experts were asked to provide corrective views on grammar, use of appropriate and intelligible words, and proper scoring scale. In addition, they were asked to rate the necessity of each item on a 3-point Likert scale (3 = necessary, 2 = useful but not necessary, and 1 = not necessary).

Then, content validity ratio (CVR) was calculated. According to the Lawshe table, items <0.51 were deleted.[17] CVR of all tool items was 0.57 and more and, thus, no item was deleted. In addition, the above 14 experts were asked to give their comments on the relevance of the scale items based on a 4-point Likert scale (1 = not relevant, 2 = relatively relevant, 3 = relevant, and 4 = quite relevant). Then, the content validity index (CVI) was calculated for each item. The items with CVI values above 0.79 were considered to be acceptable.[18] CVI of all tool items was 0.86 and greater and, thus, no item was deleted. Moreover, scale-level CVI (S-CVI) was assessed via mean scores for the item-level CVI (I-CVI). S-CVI values of >0.9 indicate that it is acceptable.[18] S-CVI of the tool was 0.92.

Scale's construct validity was evaluated with a cross-sectional study via exploratory factor analysis. The researcher received a list of all people aged 40–60 based on their electronic records from Urban Comprehensive Health Service Centers, Kashan (every citizen in Iran has an electronic health record in Urban Comprehensive Health Service Centers). Then, according to the sample size, individuals were randomly selected. Harrington argues that a sample size of 300 subjects and above is appropriate for the exploratory factor analysis.[19] In this study, the sample size was considered to be 450, but 430 subjects completed the questionnaires.

Through telephone contact with the samples, the objectives and methods of study were explained to them and verbal consent was obtained from the participants of the study. Then, the informed consent form and questionnaire were sent to the participants through WhatsApp. If a subject withdrew from the collaboration, another sample would randomly be replaced from the same center. After data collection, exploratory factor analysis was performed to extract the latent constructs of OHI-SSS. Principal components analysis (PAC) and varimax rotation were the methods of extracting latent constructs. Eigenvalue above 1, scree plot [Graph 1], and minimum factor load of 0.4 were considered for the items.

Scale's reliability was evaluated with Cronbach's alpha coefficient. The values above 0.7 were considered to be acceptable.[20] Stability was evaluated via the test–retest technique. The questionnaire was given to 20 middle-aged people twice with a 2-week interval and intraclass correlation coefficient (ICC) was calculated for the questionnaire and its subscales. ICC values above 0.75 showed acceptable stability.[21]

Data analyses were conducted with Statistical Package for the Social Sciences version 16 (SPSS Inc., Chicago, IL, USA). Descriptive statistics were used to summarize demographic characteristics. Construct validity was determined through running exploratory factor analysis. Moreover, statistical analyses including Pearson correlation analysis and paired- and independent-samples t-tests were performed. Kaiser–Meyer–Olkin (KMO) test was also used to determine the adequacy of the selected sample size for factor analysis.

This study was approved by the Ethics Committee of Kashan University of Medical Sciences, Iran (IR.KAUMS.NUHEPM.REC.1399.067). The participants signed the informed consent form in all stages of the study.

  Results Top

The mean age of the middle-aged participants was 44.1 ± 7.38 years and the average time of using the Internet was 26.14 ± 20.1 h/week. Of the 430 participants, 254 (59%) were female, 349 (81.2%) were married, and 182 (42.3%) were bachelors [Table 1]. The 24 remaining items from the previous stage entered this stage. KMO test showed the adequacy of the sample size for exploratory factor analysis (0.94). Bartlett's test of sphericity was also significant (Chi-square = 4563.965, P < 0.001) indicating a sufficient correlation between the variables [Table 2]. Factor analysis with the minimum eigenvalue >1, after varimax rotation, showed the extract of 20 items within 3 factors. Factors 1–3 explained 58.64% of the total variance [Table 3]. Factor loading of the items ranged from 0.48 to 0.81. Considering the concepts inferred from the items loaded in each factor, the first factor with 11 items was called “information reception.” The second factor was named “provide and exchange information,” which contained 6 items. The third factor with 3 items was referred to as “identification and trust” [Table 4]. Cronbach's alpha total scale was 0.93. The ICC tools and their subscales were calculated with the two-way mixed model of the absolute agreement type, which with 95% confidence, the coefficients were between 0.75 and 0.86 [Table 4].
Table 1: The participants' characteristics to evaluate the construct validity

Click here to view
Table 2: Bartlett spherical test and Kaiser–Meyer–Olkin

Click here to view
Table 3: Extracted factors with principal component analysis method

Click here to view
Table 4: Factor analysis of the Online Health Information-Seeking Skill Scale and internal reliability

Click here to view

  Discussion Top

In this study, the steps taken for the development and psychometrics of the OHI-SSS were documented according to a valid guideline.[22] The exploratory factor analysis extracted three subscales in OHI-SSS: (1) information reception, (2) provide and exchange information, and (3) identification and trust. Studies have shown that the rate of Internet use depends significantly on a person's skill.[23] Therefore, the study of users' skills to use health information online should be considered by health-care providers.

The first subscale identified in OHI-SSS was information reception. The subscale items reflect middle-aged people's capabilities in receiving health information by using different websites. The Internet has facilitated access to health information for all,[24],[25] and many people use it as the first source of information due to limited consultation time of doctors and barriers to professional health service access. While, for some people, it may be considered a supplement to health care.[25]

The second subscale measured the middle-aged people's skills in sharing their health experiences, interacting with other users, completing the online satisfaction forms related to the provided care, and engagement in health decision-making strategies. The items measured by this subscale are important issues that almost imply the user's abilities, experiences, and self-efficacy.[1] This subscale represents the potential of the Internet in offering support to a large group of online health information users.[25] The Internet allows people to communicate with health professionals as well as share their health experiences with other users.[26]

The final OHI-SSS subscale was the identification of useful online health information and trusting them. This subscale measures the skill of a middle-aged individual in identifying reliable websites of health information. Identification of health-care professionals such as specialist doctors by using reliable websites to succeed in managing one's health can be linked to the self-efficacy of individuals.[27] Online health information from unqualified websites may lead to inadequate treatment or delays in receiving health care.[28] It is, therefore, significant to devise ways for helping individuals to select valid, useful, and informative information. Health professionals may consider ways to introduce web-based health information to people.

The strength of the present scale is that, unlike other available tools, the generation of the item is based on the direct users' views, and literature review has contributed to its comprehensiveness. Despite the emergence of communication technologies and the availability of information, the use of health information is not ensured.[29] Multiple cultural and socioeconomic factors influence the extent of users utilizing online health information. Improving the delivery of health-related information necessitates a thorough understanding of users' needs. This scale provides basic knowledge about skills of related seeking online health information to policymakers so that they can make the necessary interventions. Since the respondents in this study were recruited from middle-aged people, the findings should be generalized cautiously, because the skill of using online health information varies in different age groups. The authors did not test the relationship between observed variables and their underlying structures. The authors suggest that the study be conducted for this purpose by other researchers.

  Conclusion Top

The results of this study indicated that the reliability and validity of OHI-SSS are acceptable. The Internet is an easily available source of health information, but it may create inequalities in health information accessibility among individuals who do not possess the skills of using it, especially among the elderly, those with low income, and those with low educational attainment. Therefore, there is a need to assess the skills of users, especially middle-aged people in using online health information, as this group of people will enter the stage of aging after this period, which is associated with several health challenges.


The authors would like to thank the middle-aged people who have been referred to Urban Comprehensive Health Service Centers in Kashan to participate in the study.

Financial support and sponsorship

This study was financially supported by Kashan University of Medical Sciences.

Conflicts of interest

There are no conflicts of interest.

  References Top

Kelly L, Ziebland S, Jenkinson C. Measuring the effects of online health information: Scale validation for the e-health impact questionnaire. Patient Educ Couns 2015;98:1418-24.  Back to cited text no. 1
Huh J, Kwon BC, Kim SH, Lee S, Choo J, Kim J, et al. Personas in online health communities. J Biomed Inform 2016;63:212-25.  Back to cited text no. 2
Rykov YG, Meylakhs PA, Sinyavskaya YE. Network structure of an AIDS-Denialist online community: Identifying core members and the risk group. Am Behav Sci 2017;61:688-706.  Back to cited text no. 3
Alkhatlan HM, Rahman KF, Aljazzaf BH. Factors affecting seeking health-related information through the internet among patients in Kuwait. Alex J Med 2018;54:331-6.  Back to cited text no. 4
Milligan C, Roberts C, Mort M. Telecare and older people: Who cares where? Soc Sci Med 2011;72:347-54.  Back to cited text no. 5
Tan J. E-health Care Information Systems: An Introduction for Students and Professionals. Jossey-Bass, San Francisco, CA, 2005.  Back to cited text no. 6
Turan N, Kaya N, Aydın GÖ. Health problems and help seeking behavior at the internet. Procedia Soc Behav Sci 2015;195:1679-82.  Back to cited text no. 7
Canadian Diabetes Association Clinical Practice Guidelines Expert Committee; Jones H, Berard LD, MacNeill G, Whitham D, Yu C. Self-management education. Can J Diabetes 2013;37 Suppl 1:S26-30.  Back to cited text no. 8
Pluye P, Granikov V, Bartlett G, Grad RM, Tang DL, Johnson-Lafleur J, et al. Development and content validation of the information assessment method for patients and consumers. JMIR Res Protoc 2014;3:e7.  Back to cited text no. 9
Warren E, Footman K, Tinelli M, McKee M, Knai C. Do cancer-specific websites meet patient's information needs? Patient Educ Couns 2014;95:126-36.  Back to cited text no. 10
Webb TL, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: A systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res 2010;12:e4.  Back to cited text no. 11
Razmak J, Bélanger CH. Comparing Canadian physicians and patients on their use of e-health tools. Technol Soc 2017;51:102-12.  Back to cited text no. 12
Wong DK, Cheung MK. Online health information seeking and eHealth literacy among patients attending a primary care clinic in Hong Kong: A cross-sectional survey. J Med Internet Res 2019;21:e10831.  Back to cited text no. 13
Duan W, Zheng A, Mu X, Li M, Liu C, Huang W, et al. How great is the medical burden of disease on the aged? Research based on “system of health account 2011”. Health Qual Life Outcomes 2017;15:134.  Back to cited text no. 14
Creswell JW, Clark VL. Designing and Conducting Mixed Methods Research. 3rd ed. Thousand Oaks, CA: Sage Publications; 2017.  Back to cited text no. 15
Oluwatayo JA. Validity and reliability issues in educational research. J Educ Soc Res 2012;2:391.  Back to cited text no. 16
Rose MS, Koshman ML, Ritchie D, Sheldon R. The development and preliminary validation of a scale measuring the impact of syncope on quality of life. Europace 2009;11:1369-74.  Back to cited text no. 17
Polit DF, Beck CT. Essentials of Nursing Research: Appraising Evidence for Nursing Practice. 7th ed. Philadelphia: Lippincott Williams & Wilkins; 2009.  Back to cited text no. 18
Harrington D. Confirmatory Factor Analysis. New York: Oxford University Press; 2009.  Back to cited text no. 19
Dunn TJ, Baguley T, Brunsden V. From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. Br J Psychol 2014;105:399-412.  Back to cited text no. 20
Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15:155-63.  Back to cited text no. 21
Denise F, Cheryl B. Essentials of Nursing Research: Appraising Evidence for Nursing Practice. China: Lippincott Williams and Wilkins; 2013.  Back to cited text no. 22
Yazdani-Darki M, Rahemi Z, Adib-Hajbaghery M, IzadiAvanji FS. Older adults' barriers to use technology in daily life: A qualitative study. Nurs Midwifery Stud 2020;9:229-36.  Back to cited text no. 23
  [Full text]  
Tonsaker T, Bartlett G, Trpkov C. Health information on the Internet: Gold mine or minefield? Can Fam Physician 2014;60:407-8.  Back to cited text no. 24
Chu JT, Wang MP, Shen C, Viswanath K, Lam TH, Chan SS. How, when and why people seek health information online: Qualitative study in Hong Kong. Interact J Med Res 2017;6:e24.  Back to cited text no. 25
Kirschning S, von Kardorff E. The use of the internet by women with breast cancer and men with prostate cancer-results of online research. J Public Health 2008;16:133-43.  Back to cited text no. 26
Braungart MM, Braungart RG. Applying learning theories to healthcare practice. In: Bastable SB, editor. Health Professional as Educator: Principles of Teaching and Learning. Sudbury, MA: Jones & Bartlett Learning; 2011.  Back to cited text no. 27
Cline RJ, Haynes KM. Consumer health information seeking on the internet: The state of the art. Health Educ Res 2001;16:671-92.  Back to cited text no. 28
Finney Rutten LJ, Blake KD, Greenberg-Worisek AJ, Allen SV, Moser RP, Hesse BW. Online health information seeking among US adults: Measuring progress toward a healthy people 2020 objective. Public Health Rep 2019;134:617-25.  Back to cited text no. 29


  [Table 1], [Table 2], [Table 3], [Table 4]


    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

  In this article
Materials and Me...
Article Tables

 Article Access Statistics
    PDF Downloaded129    
    Comments [Add]    

Recommend this journal