Evaluating for researchers
You can probably already deduce something about the reliability from the URL and the digital information certificate of a source/website.
| URL extension | Type of organisation | Example | Objectivity, reliability |
|---|---|---|---|
| .com | Commercial | http://www.dokteronline.com | ? |
| .net | Network | http://www.cancer.net | ? |
| .gov | Government (USA) | https://www.nlm.nih.gov | + |
| .gov(.be) | Government | https://www.ehealth.fgov.be | + |
| .edu | Educational (USA) | http://library.medicine.yale.edu | + |
| .org | International non-profit | https://nl.wikipedia.org/ | +/- |
| .be / .nl / … | Country code | https://www.kcgg.be | ? |
The use of language (e.g. correct, objective language), the proprietor (e.g. University Hospital) and the up-to-dateness will also give you additional insight into the reliability. In any case check whether you can verify the information provided in other reliable sources.
In a case-control study researchers are dealing with a group of people with a specific disease or outcome that needs investigating (Cases) and a group of people without these characteristics (Controls). The exposure to a possible risk factor is measured and compared in each of these groups. The relative risk of exposure is expressed as an Odds Ratio in a case-control study. Case-control studies are usually performed in etiological research. In the case of rare diseases or diseases that evolve over a long period, researchers can use this study set-up to look into the relationship between risk factors and the emergence of a disease. In a nested case-control study the Cases and Controls are recruited from the population of a cohort study. When, during the course of the study, sufficient patients with a specific disease (Cases) have been identified, people without the disease (Controls) are also searched for in the same cohort. Since some data from Cases and Controls are known (because of data collection in the cohort study), these can be checked for, thus reducing the chance of bias because of known disruptive variables in this set-up.
In a prospective cohort study people who may or may not have been exposed to a risk factor (such as harmful substances or a lifestyle factor) are followed up for a long period of time (usually years). The population being monitored must be free of the outcome being investigated at the beginning of the study, so that the incidence of the outcome can be calculated in the group with exposure and the group without exposure.
A cross-sectional study is a kind of observational study, in which data about risk factors and/or outcomes are collected at a specific time in a population.
Source: Van Driel M. Verklarende woordenlijst voor evidence-based medicine. 2nd revised printing Ghent: Minerva, 2003. Available online.
In a systematic review, you start with a (research) question, and conduct a search for original studies that can provide an answer to this question. The databases and methods of analysis used are referenced in the process. The results and background features of the individual studies that are found are explicitly evaluated and analysed. The standards laid down and described by The Cochrane Collaboration in the Cochrane Handbook for Systematic Reviews of Interventions are generally regarded as the gold standard for writing a systematic review. Organisations, for example The BEME Collaboration, use this methodology (for systematic reviews).
In a narrative review the author himself makes a selection of the studies to be included and there is not usually a (description of a) search strategy. As a result, the scientific value of a narrative review is limited.
Source: Van Driel M. Verklarende woordenlijst voor evidence-based medicine. 2nd revised printing Ghent: Minerva, 2003. Available online.
A meta-analysis is a systematic review in which the results of a number of comparable clinical studies are pooled and recalculated. This enables the researcher to make a more reliable claim about the effect of an intervention or treatment. Every meta-analysis is in principle preceded by a systematic review.
Source: Van Driel M. Verklarende woordenlijst voor evidence-based medicine. 2nd revised printing Ghent: Minerva, 2003. Available online.
When evaluating clinical trials, researchers use specific tools to assess the validity and reliability of the study’s findings. Two commonly used tools in this context are risk of bias tools and critical appraisal tools. Although they are sometimes used interchangeably, these tools serve distinct purposes and have different focuses.
Risk of bias tools are designed to assess the potential for systematic errors (bias) in a clinical trial that could affect the study’s outcomes. Bias can occur at various stages of the trial, from the design and data collection to the analysis and reporting. Risk of bias tools typically focus on specific aspects of the study that are prone to bias, such as randomization, blinding, and the handling of incomplete data.
For example, the Cochrane Risk of Bias (RoB) tool is commonly used to evaluate biases in randomized controlled trials (RCTs). For non-randomized studies, there are specialized tools like the Risk of Bias in Non-randomized Studies – of Interventions (ROBINS-I), which assesses biases in non-RCTs of interventions, and the Risk Of Bias In Non-randomized Studies – of Exposure (ROBINE-E), which is designed for observational epidemiological studies.
the primary objective of these tools is to identify factors that could have introduced bias, potentially distorting the study results and leading to incorrect conclusions. The choice of tool depends not only on the study type but also on the research question being addressed. For example, QUADAS-2 is tailored for assessing the risk of bias in diagnostic studies, while QUIPS is used for prognostic studies. Each tool provides a structured approach to ensure that potential biases are carefully evaluated and addressed.
Critical appraisal tools, on the other hand, provide a broader evaluation of a clinical trial’s quality. These tools not only consider the risk of bias but also assess other important aspects of the study, such as the study design, the appropriateness of the methodology, the statistical analysis, and the relevance of the findings. Critical appraisal tools are used to determine the overall quality and credibility of a study by examining both its strengths and weaknesses.
For example, the Joanna Briggs Institute provides tools tailored to a variety of study designs, including cross sectional studies, case control studies, cohort studies and qualitative research. Similarly, Critical Appraisal Skills Programme (CASP) offers ready- to-use checklists for multiple study designs, further aiding in the comprehensive assessment of research
The quality evaluation tool SYRCLE (SYstematic Review Centre for Laboratory animal Experimentation) is based on the Cochrane Risk of Bias tool, but is specifically adapted to aspects that are applicable and relevant for animal experiments.
You can assess the quality of systematic review using AMSTAR (Assessing the Methodological Quality of Systematic reviews).
The Review Manager (http://tech.cochrane.org/revman/download) software is used to create and update a Cochrane Systematic review (including the meta-analysis). The Statistics part can also be used separately, for example, to formulate a Forest Plot and carry out a meta-analysis.
The statistics software SPSS and R is available on the Athena platform. If you require statistical support, please contact the Biostatistics department in the faculty of Medicine and Health Sciences.
Via the platform Athena you can use the software NVivo.
These two presentations help you to get started:
- Self-instruction by Sofie Verhaeghe, Ineke van Eechoud en Tina Vandecasteele (based on a presentation by Saskia Decuman) (in Dutch).
- NVivo workshop by Katrien Verleye (in Dutch).
Similar software are MaxQDA, ATLAS and Dedoose.
If you would like more information or if you have any questions, you can contact our information specialist Nele Pauwels.
Do you have a question related to your literature review or do you need help with searching, processing or publishing of your literature/data? Then please fill in our form to give us a clear view of your question and so that we can advise you efficiently. Alternatively, you could email us at kcgg@ugent.be.