• Logo
  • HamaraJournals
23

Views

Evaluation Criteria for Health Websites: Critical Review

, , , and

Abstract

Introduction:

The significant usage of health websites and their roles as diagnostic and therapeutic tools have increased the importance of evaluating their credibility. Health websites are evaluated using the criteria introduced in the health guidelines; therefore, this study aimed to evaluate the adequacy of these criteria.

Material and Methods:

In this critical review study, the guidelines for "Health Websites Evaluation" and "Website Evaluation in Other Subject Areas" were extracted using sensitive keywords from valid databases, classification, comparison and content analyses were performed using scientific methods designed in this study.

Results:

The results indicate that in terms of various components of health websites, the evaluation criteria are not adequate. Note that health website evaluation criteria are designed based on the evaluation criteria of other subject areas. Therefore, the criteria share problems similar to those of the guidelines for other subject areas, and they ignore the evaluation of the specific features of health websites. It is necessary to have reliable and accurate guidelines to evaluate health websites.

Conclusion:

One of the most significant advantages of these guidelines is that using software provides an infrastructure for the automatic evaluation of health websites. Thus, the evaluation results will be available to the general public in the form of a website ranking.

INTRODUCTION

Due to powerful Internet tools, the world today is based on information [1, 2]. The use of the Internet in the health care industry has led to the provision of educational, diagnostic and therapeutic services that are high quality, low-cost, accessible and on time [3, 4]. In this atmosphere, the communication between "medical professors and students", "physicians and patients", "patients with each other" and, in particular, "health information applicants and information existing in the websites", have created special user diversity [4-7].

This platform includes a large amount of health information that is considered to be a potential opportunity to promote social health [8]. Studies have shown that people in every age group search the Internet for health information and use their findings [9]. Furthermore, a variety of texts, images, graphics, audio and video files and applications from different agencies, including governments, hospitals, universities, research centers, medicine manufacturers, medical equipment manufacturers, business organizations, public associations and non-accredited organizations, are loaded on various websites. This information bombardment distorts the minds of health information seekers and affects appropriate decision-making [10]. Note that users' lack of medical skills affects their search for information on the Internet which, along with their low level of health literacy (which reduces the quality selection of authentic and credible sources and services) is a serious problem [11].

However, physicians and health care providers pay attention to the accuracy of information uploaded on the website and the provision of high quality services of eHealth. These providers would also be eager to benefit from the advantages of these virtual environments as an effective assistant, if the concerns are addressed [12]. Therefore, considering the importance of obtaining accurate and high quality health information, it is necessary to validate websites and rank their confidence levels [13]. To achieve this purpose, health websites evaluation is an attainable executive solution [13-15]. In this regard, a number of credible organizations have produced guidelines, including health evaluation criteria [8, 14, 16]. Some of these guidelines include the Health On the Net (HON) foundation [17], Medical Library Association (MLA) [18], MedlinePlus (NLM) [19], National Center for Complementary and Integrative Health (NCCIH) [20] and Food and Drug Administration (FDA) [21].

Currently, websites that use specific components, have been developed utilizing the most up-to-date technologies, and based on their services, they have been classified into different types [22]. As the result of technological progress, health websites have experienced tangible usage changes compared with other websites; thus, to evaluate them, the following principal question must be asked, "Are the evaluation criteria presented in current guidelines adequate and updated?"

To answer this question, no in-depth comparative study has addressed the status, coverage, and adequacy of health evaluation criteria. Thus, this study aimed to determine their strengths and weaknesses. Considering the specific features and characteristics of these websites, a comparison was made between health websites evaluation criteria and the website evaluation criteria of other subject areas.

MATERIAL AND METHODS

In this critical review, the preferred reporting items for systematic reviews of PRISMA guidelines were used as models, and their generalities were used.

Information resources

A variety of databases and search engines, including PubMed, ScienceDirect, Web of Science, ProQuest and Google Advance, were searched, according to the below-mentioned pre-specified search strategy.

Keywords

A list of the search terms commonly used for website evaluations was obtained from published literature. These terms included Evaluation, quality, health information, credibility, reliability, accuracy, readability, Criteria, Website, Internet, electronic and eHealth.

Search strategy

The search query was: (quality OR credibility OR reliability OR accuracy OR readability OR evaluation OR assessment) AND (health information) AND (online OR Internet OR web OR eHealth OR e-Health OR cyber* OR electronic) AND (criteria OR criterion).

Inclusion criteria

Full-text resources were available.

The guidelines were up-to-date.

The guidelines were defined through credible universal references.

The evaluation criteria were presented in the guidelines.

Exclusion criteria

There were not enough evaluation criteria in the guidelines.

The definitions were ambiguous and irrelevant to the evaluation criteria used in the guidelines.

In relation to evaluation principles, the guidelines were not comprehensive.

Resource extraction

The search results are shown in Fig 1. Resource extraction was performed using the following methods:

Forty-three resources from PubMed, 55 from ScienceDirect, 32 from Web of Science, 38 from ProQuest and 190 from Google Scholar were retrieved. After reviewing the titles and abstracts of the retrieved resources, 75 recurrent references were excluded. Then, 283 resources were carefully studied and compared with the inclusion criteria of this study. In terms of the exclusion criteria, the remaining 74 sources were studied by the researchers. During this process, 20 guidelines were selected for analysis.

FIH-9-44-g001.jpg

Fig 1

Literature search criteria with inclusion and exclusion criteria

Content Categorization and Analysis

Content categorization

Guidelines were categorized into two parts based on the nature of their usage. According to their nature, the guidelines were categorized into two groups. The first group included health website evaluation guidelines, which were named "Health Guidelines". The features of these guidelines are presented in Table 1.

The second group contains website evaluation guidelines in other subject areas, which were named "Other Guidelines". A list of features of these guidelines is presented in Table 2.

Content analysis

Study of the evaluation criteria

Because the names and definitions of the evaluation criteria used in the guidelines differed, for subsequent comparisons, it was necessary to assign unified names and definitions for each evaluation criterion. For this purpose, in each of the groups (including "Health Guidelines" and "Other Guidelines"), the following steps were taken.

First, the criteria that seemed to be conceptually similar and shared many common features were placed in the same category. In each of these categories the criteria were examined. Regarding the repeatability of the names assigned to the criteria having correct meanings, a proper name was chosen for each criterion. The results are shown in Table 3.

Then, based on the definitions that were commonly associated with each one of the criteria, a unified definition of the criterion's concept was proposed. The results of the definitions are shown in Table 4. Note that to present unified definitions, attention was directed toward the common points between definitions, and no attempt was made to provide accurate and perfect definitions.

Analysis of the evaluation criteria in "Health Guidelines"

Considering the specific features and characteristics of health websites, the adequacy of the evaluation criteria in the group of "Health Guidelines" was examined.

Comparative analysis of the evaluation criteria in the two groups of guidelines

Multiple use of each evaluation criterion in both guideline groups, including the group of "Health Guidelines" and the group of "Other Guidelines", was separately examined, and the results are shown in Table 5 and 6.

Multiple evaluation criteria in one group of guidelines compared with the second group were reviewed, and the results related to the status of health websites were reported.

Table 1

Features of health guidelines

Reference Time Of Updating Abbreviation Name Guideline Code Row
[17] 2013 HON Health On the Net Foundation H01 1
[18] MLA Medical Library Association H02 2
[19] 2015 NLM MedlinePlus National Library of Medicine H03 3
[20] 2017 NCCIH National Center for Complementary and Integrative Health H04 4
[23] 2016 Dalhousie University's Kellogg Health Science Library H05 5
[24] 2014 University of Maryland University College H06 6
[25] 2016 Healthdirect Australia H07 7
[21] 2016 FDA Food and Drug Administration H08 8
[26] 2016 American Cancer Society H09 9
[27] 2017 Office of Disease Prevention and Health Promotion H10 10

Table 2

Features of other guidelines

Reference Time Of Updating Name Guideline Code Row
[28] 2017 Cornel University Library O01 1
[29] 2017 RIO SALADO COLLEGE LIBRARY O02 2
[30] 2016 UNIVERSITY LIBRARY UNVERSITY OF ILLIONOIS URBANA CHAMPAIGN O03 3
[31] 2011 TRIO Training UNIVERSITY of WASHINGTON O04 4
[32] 2017 ‌Berkeley library UNIVERSITY of CALIFORNIA O05 5
[33] 2017 PURDUE LIBRARY O06 6
[34] 2017 Harvard Guide to Using Sources O07 7
[35] 2017 University library of Virginia Polytechnic O08 8
[36] 2017 Kent State University Libraries O09 9
[37] 2015 THE UNIVERSITY OF EDINBURGE O10 10

RESULTS

The large number of evaluation guidelines [8, 38-40] indicated that there is a possible shared common fundamental structure, and it seems that this structure has been developed by researchers in light of the technological advances. According to the findings of the present study, no evolutionary growth is visible in these guidelines. Additionally, these guidelines are provided with individual, personal or organizational views and do not follow a valid and general instruction. The following reasons might explain this premise.

Names dispersion

As shown in Table 3, numerous names were assigned to a virtually common concept associated with a criterion. Therefore, the names do not have a specific root. This problem is clearly visible in "Health Guidelines" and "Other Guidelines".

Table 3

Features related to the names of evaluation criteria

Row Criterion Code Selected English Name for Evaluation Criterion Other English Names Attributed to the Criteria in 'Health Guidelines' Other English Names Attributed to the Criteria in 'Other Guidelines'
1 C01 Authority Credibility, Who?, Who sponsors?, sponsors, Authoritative, Who is sponsoring?, Consider the source, Who wrote?, Who runs?, Who runs this website?, Who pays?, Transparency, The names of the site developers, partners, and funding sources Author/Creator, Author's credentials, Publishers, Domain, What is the site's domain?, The URL
2 C02 Reliability The information is reliable Reliable sources
3 C03 Currency How often is the site updated? Date of Publication, Timeliness
4 C04 Functionality Does the page look professional and function well?
5 C05 Privacy Protect your privacy, How does the site handle personal information? Is your privacy respected? What information about its visitors the site collects? Does the website ask for your information?
The site protects users' personal information
6 C06 Purpose What will “they” do with it? Why? The site purpose, The purpose of the site, What to do? What's the purpose or mission?
7 C07 Disclosure Financial disclosure
8 C08 Usability Interaction, How does the site manage interactions with users? How the site manages communications with visitors?
How do users interact with this website?, Design, Organization, Ease of Use, The site is well maintained
How did you navigate to this Page?
Design, Ease of use / Special features
9 C09 Accessibility Accessibility
10 C10 Audience Who is the intended audience?
What do they want from you? Audience of the site, It is intended audiences are clearly stated?
Intended audience, Level of Information,Relevancy
11 C11 Content Content
12 C12 Coverage Complete
13 C13 Bias Justifiability, Does the site present facts and not opinion? Beware of bias,
Be a cyber-skeptic, Is it balanced?, Advertising policy, Advertisement labels clearly identify and separate ads from health information
Objectivity, Is the information biased?
14 C14 Accuracy Is it accurate? Focus on quality, Original source(s) of health information, Is it clear? Quality of Publication
15 C15 Citation Attribution, How is it? documented,Source Sources Cited, Documentation
16 C16 Link What is the site's policy about linking to other sites? Link to local home page

General, ambiguous and simple definitions

In some "Health Guidelines", definitions are not clear enough, and in some of them, such as "Health On the Net Foundation", the definitions are too general. For example, in this guideline, the authority criterion is defined as "The qualifications of the authors" [17]. Other criteria for this guideline are general in the same manner. In a number of "Other Guidelines", definitions are presented as questions that do not lead to the formation of a precise concept of criterion in the mind of the evaluator. One of these guidelines is "Health Direct Australia" [25]. For the mentioned reasons and also due to the lack of unified definitions in the resources, to make a comparison between different criteria, unified definitions were created, as shown in Table 4.

In some cases, in the "Health Guidelines", a single criterion, based on the definitions provided in Table 4, includes the definitions of several criteria.

The definitions provided in Table 4 have led to certain notions that form the basis for the judgments in this research. In some instances, based on these definitions, interventions between the concepts of health guideline criteria have been observed. For example, in "MedlinePlus", purpose also conveys the concept of bias, and they cannot be differentiated from one another [19]; in the "National Center for Complementary and Integrative Health", within the definition of the authority criterion, a definition of bias criterion is also presented [20].

Table 4

Definitions related to the evaluation criteria

Reference Unified Definitions for Concepts Associated with the Evaluation Criteria Selected Name for Evaluation Criterion Criterion Code Row
[17, 41-43] Authority refers to the study of qualification, identity of the organization, or the custodians of a certain website and authors of the resources. This criterion evaluates the ethics, legality and validity of the information. Communication of the website custodians through addresses, phone numbers and email addresses can be evaluated through this criterion. Authority C01 1
[17, 44] Reliability examines the correctness of the operations while repeating the similar process on a website. Getting similar operations results while executing repetitive activities is important. The ISO standard considers this criterion for controlling error tolerance, the ability to retrieve information and the completeness of the information content while the processes are being repeated. Reliability C02 2
[18, 21, 42, 45] Currency examines the information updating time on a website. To be aware of timeliness, the time of creating the website and the latest time of updating the information should be considered. Currency C03 3
[46, 47] Functionality examines the technical and technological output of a website, including speed, settings, alerts, reminders and search ability. Website performance can be examined through this criterion. Functionality C04 4
[17, 42, 46] Privacy examines the security and confidentiality of a website. This examination is important in two respects: A) to prevent information theft by hackers, and b) to ensure that website custodians do disclose their users' private information. Privacy C05 5
[19, 21, 44, 48] Purpose examines the goals of creating a website and providing resources. The reason for a website's activity specifies the website type. Purpose C06 6
[19, 21, 49] Disclosure examines the revenue-generating purpose and the associates who are affected by the financial benefits of a website. Disclosure C07 7
[17, 19, 43, 47] Usability examines the ease of use and user satisfaction while working on a website. Having help options and settings for changing font, color, background textures and page width are some of the items that improve the usability of a website. Usability C08 8
[19, 20, 42, 46] Accessibility examines the availability of a user's required information, such as the relevant information of products. Accessibility C09 9
[21, 48, 50] Audience examines the existence of classified information of a website based on different groups of users, with or without health literacy. Audience C10 10
[51, 52] Content examines the correctness, accuracy, timeliness, usefulness, clarity and observing legal notices on a website. Content C11 11
[41, 44, 45] Coverage examines the existence of all information needed by a user in relation to the subject of a website. The concept of coverage lies in the meaning of completeness associated with the purpose of the website. Coverage C12 12
[17, 44] Bias examines the impartiality and neutrality of the information published on a website. The focus of this criterion is on identifying non-commercial and impartial information and preventing deviation from the truth. Bias C13 13
[17, 19, 44, 50] Accuracy examines the correctness of the information available on a website and determines the validity of the information. Accuracy C14 14
[17, 21, 51] Citation examines the existence of references for the scientific information. Therefore, the scientific nature of the resources is determined through links. Citation C15 15
[19, 21, 49] Link, as a criterion, examines the accessibility of information suggested by the custodians of a website for further reading on other websites. These resources should be relevant to the purpose of the website, and they should be up-to-date. It is important that if there is a link, it should be active. Link C16 16

In some cases, within the "Health Guidelines", two different criteria, based on the definitions in Table 4, share similar concepts. In some instances, there are resembling definitions between two distinct criteria in the guidelines. For example, the definition of bias in "Dalhousie University's Kellogg Health Science Library" is the same as the definition of the authority in the "American Cancer Society" Guideline, which is referred to as "Who runs this website?" [23, 26].

Differences of selected evaluation criteria in different "Health Guidelines"

In Table 5, columns represent guideline codes and rows indicate selected criteria. It is obvious that a criterion is used in a specific number of guidelines. The results of this table indicate that in each guideline, a group of criteria have been selected, according to the supplier's opinion, and some of the criteria have been neglected. Some significant criteria that are essential for evaluating health websites have been ignored, and the reason for this neglect is unknown. Note that authority and currency have been of particular importance among the nine selected guidelines. However, privacy and purpose, which are particularly important for evaluating health websites, are not selected in three guidelines, and in four guidelines, accuracy and bias have not been taken into account. Audience has not been included in five guidelines. In addition, reliability, disclosure, usability, content, citation and link are rarely chosen.

Table 5

Comparison of evaluation criteria in "Health Guidelines" Group.

H10 H09 H08 H07 H06 H05 H04 H03 H02 H01 Guideline Code
Criterion Name
* * * * * * * * * Authority
* Reliability
* * * * * * * * * Currency
Functionality
* * * * * * * Privacy
* * * * * * * Purpose
* Disclosure
* * Usability
Accessibility
* * * * * Audience
* Content
Coverage
* * * * * * Bias
* * * * * * Accuracy
* * Citation
* Link

Table 6

Comparison of Evaluation criteria in "Other Guidelines" Group

Guideline Code O01 O02 O03 O04 O05 O06 O07 O08 O09 O10
Criterion Name
Authority * * * * * * * * * *
Reliability * *
Currency * * * * * * * * * *
Functionality *
Privacy
Purpose * * *
Disclosure
Usability * * * *
Accessibility *
Audience * * * * *
Content * * *
Coverage * * * * * *
Bias * * * * * * * * *
Accuracy * * * * * * * * *
Citation * *
Link *

Lack of a questionnaire

Although there are questionnaires for evaluating health websites in two guidelines HON foundation and college of Maryland university [17, 24], in eight "Other Guidelines", no precise questionnaire was found. In particular, if there is a questionnaire, the evaluation method of a website is determined. The absence of a questionnaire has made some researchers customizing questionnaires to evaluate health websites [53, 54], which has led to different reports of website evaluation results by different groups.

Unclassified evaluation criteria

In some studies of other subject areas, some criteria, including different groups of content, design, organization and usability, are presented hierarchically [9, 48], which contributes to the depth of the website evaluation, resulting in discovering dimensions of the strengths or weaknesses of the website [9]. In the ten surveyed guidelines, this hierarchical view was not found. As shown in Table 4, these criteria are better classified into different groups. For instance, the definition of content in Table 4 contains two other criteria: currency and accuracy. Therefore, content can be a group name, and currency and accuracy are its objects.

Absence of weighted evaluation criteria

In none of the studies were the criteria weighted based on their degree of importance. Some of the criteria, compared to others, examined important aspects of a website's status, and they have more evaluation impact that should be specified.

However, the criteria presented in the "Health Guidelines" seem to be designed based on "Other Guidelines". In support of this claim, the following items can be mentioned:

"Other Guidelines" have a history, and website evaluations are initially performed in other subject areas, particularly in library science [55].

All existing problems in "Health Guidelines" are also found in "Other Guidelines". For example, the findings shown in Table 6 show the dispersion of the selection criteria in the "Other Guidelines", which were similar to the findings discussed in Table 5.

The results from the comparison of Table 5 and 6 illustrate that the criteria in ten "Health Guidelines", excluding functionality, accessibility and coverage, also exist in "Other Guidelines".

DISCUSSION

There is currently no consensus for the definition of eHealth in available resources [56]. In a 2001 article, titled "The eHealth landscape: a terrain map of emerging information and communication technologies in health and health care," Eng stated that transferring health information with the aim of improving the quality of health services, by the use of Internet is a good definition for it [57]. In a 2002 article entitled, "Personalizing medicine on the Web. E-health offers hospitals several strategies for success," Meyer and colleagues declared that services provided by eHealth can be categorized into four main groups: eHealth commerce, eHealth content, eHealth care and eHealth connectivity [58]. In a 2002 article entitled, "Kundenbindungsstrategien von e-Health Services-Anbietern," Kirchgeorg and Lorbeer proposed that in addition to the four abovementioned groups, eHealth communities should also be taken into account [45]. To supply these services, there are a variety of health website types, including medical websites, health portals, documental websites and information websites are used [22]. The quality of health websites seems to depend on the quality of the services that they provide [56]. Therefore, evaluating the quality of services provided by health websites determines the degree of their validity.

Note that the provision of such services can be carried out through various components. For instance, a medical website, providing medical services can use such components as teleconference, chat or electronic records [59]. A patient portal needs components to send and receive information between patients, health care providers and electronic records [60, 61].

The subjects mentioned above suggest that a simple look at the evaluation of these widespread services, which has led to the creation of a variety of health websites, would not yield valuable results. This study found that the criteria in "Health Guidelines" are taken from "Other Guidelines" criteria, and they are not appropriate for evaluating various components in health websites. In this regard, in a 2016 article entitled, "Making Quality Health Websites a National Public Health Priority: Toward Quality Standards," Devine and colleagues acknowledged that, given the key role of health websites, there is no standardized and accurate criterion to evaluate them [9]. In a 2010 article entitled, "Revisiting the online health information reliability debate in the wake of "web 2.0": an inter-disciplinary literature and website review," Adams likewise emphasized that for the evaluation of health websites, which have been transformed by technological changes and are providing new services, guidelines have not been updated [62].

According to some "Health Guidelines", the scoring systems of websites have been developed as "Quality Evaluation Tools", including HON Code, DISCERN, and JAMA benchmarks [63]. Not being new and having some difficulties, these tools are utilized to evaluate health websites [39]. Currently, the most comprehensive tool for measuring the quality of health websites, WebMedQual, evaluates specific dimensions of health websites [64]. However, because important dimensions, such as accuracy and accessibility, are not evaluated by this software, it is not considered to be a perfect tool [64]. The incompatibility and evolution of these tools, in relation to the development of health website services, are obvious [65]. However, it seems that most of the problems with these tools are related to the shortcomings of the evaluation criteria in the guidelines already discussed in this study.

The compatibility of the evaluation criteria in the guidelines with the components used in the website should not be ignored. In this study, only one guideline (named "e-Health Code of Ethics") that defines the evaluation criteria of health websites compatible with recent components of health websites and addresses their differences was found. For example, in the content evaluation group, this guideline has distinctly emphasized medical information, information on medical results or information about the health care providers as different criteria. Additionally, commerce, privacy, care and other criteria, including their features, have been proposed. This guideline focused on the criterion of consultation with a specialist and how to communicate through website components, such as email or other facilities of a website [66].

Patients are not the only users of health websites; physicians and health care providers as specialists also pay attention to the quality of online services [12]. Thus, true comprehension of health websites evaluation will lead to valuable results. Therefore, the following steps are recommended to design a systematic approach to evaluate websites.

First, it is necessary to describe eHealth services and the classification of health websites based on their services. Due to the disagreement among researchers, it is essential to conduct an in-depth study and come to a consensus [45, 58, 67]. In fact, the classification of health websites based on their services helps evaluators identify required components, apply appropriate evaluation criteria according to the type of components and evaluate the existence and adequacy of these components. For example, the components used in a patient portal as a type of website, have commonalities and differences with components used in a telemedicine website. These commonalities and differences have to be precisely defined, so that at the time of evaluation, the nature of the website and its functionality can be evaluated.

Afterwards, it is possible to introduce evaluation criteria based on each service group. Thus, a package of evaluation criteria associated with each electronic service can be provided. Therefore, if the specific service on any type of a website is usable, the relevant package can be called and used for evaluation. By providing these packages, it is necessary to solve the problems associated with the evaluation criteria that are discussed in this study. In particular, it is significant to determine a quantity scale for all the criteria and specify their weights. Finally, it is possible to design a comprehensive guideline with coherent evaluation methods.

The following benefits can be obtained from the steps mentioned above:

If new services are created in light of technological advances, the evaluation criteria for these services will be specified; thus, previous services and its evaluation criteria do not alter.

If there is a primary agreed method for evaluating each service, it can be gradually evolved. As a result of this evolution, the evaluation of the websites providing these services will be more accurate.

During evaluation, despite having reliable guidelines, the website type is identified, and based on its functionality and usability, the website is ranked and presented to the general public.

CONCLUSION

Health websites play a key role in providing health services and promoting social health. Therefore, the general public, including healthy people, patients, and particularly health care providers, are considered to be users of these websites. The remarkable usage of these websites and their intervention as diagnostic and therapeutic tools adds value to their evaluation when determining their degree of validity. This study found that the evaluation criteria of these websites, which are in "Health Guidelines", are not adequate enough to evaluate these new technologies. In addition, because there are no comprehensive and standardized guidelines for evaluating health websites [68], no meaningful comparison of the evaluation results can be concluded [64].

Obviously, if there is a reliable and accurate guideline for evaluating health websites, standards can be defined for designing these websites, and the infrastructure for automatically evaluating them can be provided by implementing evaluation software as robots; then, the evaluation results of these robots would be standardized in each evaluation and quickly updated based on website changes. Ultimately, the website type is identified, and based on its functionality and usability, it is ranked and presented to the general public.

ACKNOWLEDGEMENTS

This study was supported by Iran University of Medical Sciences.

AUTHOR’S CONTRIBUTION

The authors agree on this final form of the manuscript, and attested that all authors contributed in the final draft of the manuscript. 

CONFLICTS OF INTEREST

The authors declare no conflicts of interest regarding the publication of this study.

FINANCIAL DISCLOSURE

No financial interests related to the material of this manuscript have been declared.

References

1. Hewson, C.; Stewart, DW. Internet research methods. Wiley Online Library; 2016.
2. Houck DA, Kraeutler MJ, Belk JW, McCarty EC, Bravman JT. Evaluation of information available on the internet regarding reverse total shoulder arthroplasty. Shoulder Elbow 2019;11(2 Suppl):29–34.
3. Hilty DM, Ferrer DC, Parish MB, Johnston B, Callahan EJ, Yellowlees PM. The effectiveness of telemental health: A 2013 review. Telemed J E Health 2013;19(6):444–54.
4. Gomes AW, Butera G, Chretien KC, Kind T. The development and impact of a social media and professionalism course for medical students. Teach Learn Med 2017;29(3):296–303.
5. Greene JA, Choudhry NK, Kilabuk E, Shrank WH. Online social networking by patients with diabetes: a qualitative evaluation of communication with facebook. J Gen Intern Med 2011;26(3):287–92.
6. Guo S, Guo X, Zhang X, Vogel D. Doctor–patient relationship strength’s impact in an online healthcare community. Information Technology for Development 2018;24(2):1–22.
7. Moorhead SA, Hazlett DE, Harrison L, Carroll JK, Irwin A, Hoving C. A new dimension of health care: Systematic review of the uses, benefits, and limitations of social media for health communication. J Med Internet Res 2013;15(4):e85.
8. Kim P, Eng TR, Deering MJ, Maxfield A. Published criteria for evaluating health related web sites: Review. BMJ 1999;318(7184):647–9.
9. Devine T, Broderick J, Harris LM, Wu H, Hilfiker SW. Making quality health websites a national public health priority: Toward quality standards. J Med Internet Res 2016;18(8):e211.
10. Graham, AL.; Cobb, CO.; Cobb, NK. The Internet, social media, and health decision-makingIn: Diefenbach M, Miller-Halegoua S, Bowen D. , editors. Handbook of health decision science. New York: Springer; 2016.
11. Gutierrez N, Kindratt TB, Pagels P, Foster B, Gimpel NE. Health literacy, health information seeking behaviors and internet use among patients attending a private and public clinic in the same geographic area. J Community Health 2014;39(1):83–9.
12. Bernstam EV, Shelton DM, Walji M, Meric-Bernstam F. Instruments to assess the quality of health information on the World Wide Web: What can our patients actually use?. Int J Med Inform 2005;74(1):13–9.
13. Brann M, Anderson JG. E-medicine and health care consumers: recognizing current problems and possible resolutions for a safer environment. Health Care Anal 2002;10(4):403–15.
14.The Health Improvement Institute. Consumer WebWatch. A report on the evaluation of criteria sets for assessing health web sites. Health Improvement Institute. Bethesda; 2003.
15. Hoffman-Goetz L, Clarke JN. Quality of breast cancer sites on the World Wide Web. Can J Public Health 2000;91(4):281–4.
16. Zeraatkar D, Johnston BC, Guyatt G. Evidence collection and evaluation for the development of dietary guidelines and public policy on nutrition. Annu Rev Nutr 2019;39:227–47.
17. Bastos BG, Ferrari DV. Internet and education for the patient. Arquivos Internacionais de Otorrinolaringologia 2011;15(04):515–22.
18. Roberts L. Health information and the Internet: The 5 Cs website evaluation tool. Br J Nurs 2010;19(5):322–5.
19. Cline RJ, Haynes KM. Consumer health information seeking on the Internet: The state of the art. Health Educ Res 2001;16(6):671–92.
20. Aly MA-S. An evaluative study of some online websites for learning and teaching english as a foreign language [MSc Thesis] Faculty of Education, Benha University. Egyp 2008;
21. Bremner JD, Quinn J, Quinn W, Veledar E. Surfing the net for medical information aboutpsychological trauma: An empirical study of thequality and accuracy of trauma-related websites. Med Inform Internet Med 2006;31(3):227–36.
22. Giacomo PD, Maceratini R. Health websites in Italy: use, classification and international policy. Med Inform Internet Med 2002;27(3):153–60.
23.
citation-type="Government"
24.
citation-type="Government"
25.
citation-type="Government"
26.
citation-type="Government"
27.
citation-type="Government"
28.
citation-type="Government"
29.
citation-type="Government"
30.
citation-type="Government"
31.
citation-type="Government"
32.
citation-type="Government"
33.
citation-type="Government"
34.
citation-type="Government"
35.
citation-type="Government"
36.
citation-type="Government"
37.
citation-type="Government"
38. Jadad AR, Gagliardi A. Rating health information on the Internet: navigating to knowledge or to Babel?. JAMA 1998;279(8):611–4.
39. Gagliardi A, Jadad AR. Examination of instruments used to rate quality of health information on the internet: Chronicle of a voyage with an unclear destination. BMJ 2002;324(7337):569–73.
40. Coulter A, Ellins J, Swain D, Clarke A, Heron P, Rasul F, et al. Assessing the quality of information to support people in making decisions about their health and healthcare. Picker Institute Europe, Oxford; 2006;
41. Selim HM. Content evaluation criteria for general websites: Analysis and comparison. International Journal of Online Marketing (IJOM) 2012;2(3):21–38.
42. Nădăşan V. The quality of online health-related information–an emergent consumer health issue. Acta Medica Marisiensis 2016;62(4):408–21.
43.
citation-type="Government"
44. Singh SP. Evaluation of electronic reference sources. DESIDOC Journal of Library & Information Technology 2003;23(2):43–47.
45. Kirchgeorg, M.; Lorbeer, A. Kundenbindungsstrategien von e-Health services-anbietern. Bruhn M, Stauss B. , editors. Gabler Verlag, Wiesbaden: Electronic Services; 2002.
46. Zeid, H.; Handzic, M. International Symposium on Sustainable Development. Sarajevo, Bosnia and Herzegovina; 2012. Commercial websites evaluation.
47. Mebrate, TW. A framework for evaluating academic website’s quality from students’ perspectiveIn: Delft, University of Technology; Delft, . , editors. 2010.
48. Hasan L, Abuelrub E. Assessing the quality of web sites. Applied Computing and Informatics 2011;9(1):11–29.
49. Association APH. Criteria for assessing the quality of health information on the Internet. Am J Public Health 2001;91(3):513–4.
50.
citation-type="Government"
51.
citation-type="Government"
52. Cui, S.; Reynolds, C.; McGowan, J. The Internet as a legal research toolIn: Saint-Onge M, Corbelli L. , editors. For law library decision-makers. American Association of Law Libraries; 2000.
53. Rennie CA, Hannan S, Maycock N, Kang C. Age-related macular degeneration: What do patients find on the internet?. J R Soc Med 2007 ;100(10):473–7.
54. Minzer-Conzetti K, Garzon MC, Haggstrom AN, Horii KA, Mancini AJ, Morel KD, et al. Information about infantile hemangiomas on the Internet: How accurate is it?. Journal of the American Academy of Dermatology 2007;57(6):998–1004.
55. Urgo M. Deconstructing Web sites: An alternative technique for Web site evaluation and filtering. Business Information Review 1997;14(2):85–7.
56. Hadwich K, Georgi D, Tuzovic S, Büttner J, Bruhn M. Perceived quality of e-health services: A conceptual scale development of e-health service quality based on the C-OAR-SE approach. International Journal of Pharmaceutical and Healthcare Marketing 2010;4(2):112–36.
57. Eng TR. The eHealth landscape: A terrain map of emerging information and communication technologies in health and health care. Robert Wood Johnson Foundation Princeton, NJ; 2001;
58. Meyers J, Van Brunt D, Patrick K, Greene A, editors . Personalizing medicine on the Web: E-health offers hospitals several strategies for success. Health Forum J 2002;45(1):22–6.
59. Lowery JC, Hamill JB, Wilkins EG, Clements E. Technical overview of a web-based telemedicine system for wound assessment. Adv Skin Wound Care 2002;15(4):165–9.
60. Irizarry T, Dabbs AD, Curran CR. Patient portals and patient engagement: A state of the science review. J Med Internet Res 2015;17(6)
61. Davis SE, Osborn CY, Kripalani S, Goggins KM, Jackson GP. Health literacy, education levels, and patient portal usage during hospitalizations. AMIA Annu Symp Proc 2015;2015:1871–80.
62. Adams SA. Revisiting the online health information reliability debate in the wake of “web 20”: An inter-disciplinary literature and website review. Int J Med Inform 2010;79(6):391–400.
63. Fahy E, Hardikar R, Fox A, Mackay S. Quality of patient health information on the Internet: Reviewing a complex and evolving landscape. Australas Med J 2014;7(1):24–8.
64. Provost M, Koompalum D, Dong D, Martin BC. The initial development of the WebMedQual scale: Domain assessment of the construct of quality of health web sites. Int J Med Inform 2006;75(1):42–57.
65. Howitt A, Clement S, de Lusignan S, Thiru K, Goodwin D, Wells S. An evaluation of general practice websites in the UK. Family Practice 2002;19(5):547–56.
66.
citation-type="Government"
67. Skorin-Kapov L, Matijasevic M. Analysis of QoS requirements for e-health services and mapping to evolved packet system QoS classes. International Journal of Telemedicine and Applications 2010;2010
68. Maloney S, Ilic D, Green S. Accessibility, nature and quality of health information on the Internet: A survey on osteoarthritis. Rheumatology (Oxford) 2005;44(3):382–5.

This display is generated from Gostaresh Afzar Hamara JATS XML.

Refbacks

  • There are currently no refbacks.