Overview

Article appraisal is an important activity that determines the relevance, value, and trustworthiness of research evidence to guide professionals in the efficient and effective use of the evidence (Mhaskar et al., 2009). Hence, the current paper is an appraisal of an article, “Outcomes of Obese, Clozapine-Treated Inpatients with Schizophrenia Placed on a Six-Month Diet and Physical Activity Program” by Wu, Wang, Bai, Huang, and Lee  published in the year 2009 in the Psychiatric Services journal. The purpose of the current paper is to determine the validity of results and aims to establish the causal association between antipsychotic drugs, namely, clozapine, and weight gain. The paper begins by providing a background rationale for the goal of the study, which is increased appetite that leads to weight gain among schizophrenia patients on antipsychotic drugs that is the opening remark. However, such an ultimatum should have been referenced because it is not the authors’ own thoughts as it stems from research findings. The drug selected is described and the selected process justified. Scientific literature has been given to justify the effects of clozapine, which have influenced its selection. The issue being addressed is important considering that obesity is both a cosmetic and health concern, prevailing as a risk factor for chronic illnesses, such as diabetes, heart diseases, and hypertension (Çakmur, 2017).

The study’s goals are clear towards the end of the introduction, but the objectives are missing. There is a difference between the general objective (overall goals/aims/purposes) and specific objectives (Open University, 2017). The specific objectives are developed from the general objectives and are SMART, focusing on the what, where, how, and when of the study. The study adopted a randomized, controlled trial, which is regarded as the gold study and the best research design that a research study could ever adopt (Guerrera, Renaud, Tabbò, and Filosso, 2017). The study is not informative to a reader because whereas it points out that there are two study groups, these have not been well articulated. Thereby, the reader is left to make his or her own judgment, and a layperson would not understand the article. Only a person with a good background knowledge about randomized controlled trials would understand the placement of the study groups. The intervention group was comprised of individuals on diet control and physical activity while the control was presumably on standard care in the chosen healthcare facility. The intervention was carried out by a dietitian and surgeon where the dietitian placed women in the intervention group on a 1,300 to 1,500 calorie diet each day and men on a 1,600 to 1,800 kcal diet each day. The proportion of the macronutrients were 20%, 25%, and 55% for protein, fat, and carbohydrates, respectively which are compliant with recommendations for a heart diet. Also, the intervention group had to undertake physical activity for at least 30 minutes three days every week. As mentioned earlier, the authors have not clearly described the study population, but they provided the overall sample size without stating how it was calculated. Also, the sampling approach used to obtain the study population from which the sample was drawn is not provided. Guerrera et al. (2017) state that a precise statistical connotation should be linked to the study as the sample size should be adequate to meet the power needed to ascertain the internal validity of the results. There are three categories of the study measures: blood sampling, anthropometric measures, and metabolic analysis and enzyme immunoassay.

Most of study’s results were well explained and began by indicating the final sample size that was used. However, there are some results missing in relation to compliance with the stipulated guidelines and standards guiding the study: magnitude of physical activity among, and diet quality of, the participants. Also, the findings have not been well presented because there is no good alignment between the content and the tables used to augment the explanations. Whereas the tables should be part of the results section, one can see tables in the discussion section, which is merely a discussion of results using previous researches. Body fat was not significantly different within and between the control and intervention groups at three and six months, and neither was that of men and women who had differing body fat percentages and fat free masses. However, there was a significant decrease in the BMI of the intervention group. Waist circumference and body weight reduced significantly at three and six months in the intervention group whereas the hip circumference reduced significantly at the sixth month. Significant change in biochemical parameters were observed for cortisol, insulin, and triglyceride where significant increases in triglyceride levels were seen after the third month in the control group and significant increases in insulin levels were seen in the intervention group after the sixth month. Significant decreases for triglycerides and cortisol were reported in the intervention group at the different study periods. The authors have not provided relevant and useful directions for future research related to the topic.

Internal Validity

There are various factors that affect internal validity as developed by Campbell and Stanley (1996). Based on these factors, the current study can be deemed to have medium internal validity because some of these factors do not play a role in the current study while others do play a role, as discussed by Taylor and Asmundson (2007). There is a likelihood that the results were likely to be by chance because the study did not accurately account for the intervention in reference to dosage. Various biases that might interfere with internal validity are indicated below.

The sample size is one facet that helps to ascertain the validity of results. In the current study, this parameter is not clear because the means by which the sample size was obtained are not clear and the statistical power related to an adequate sample size is not provided. Thereby, the adequacy of the sample size is not known. Guerrera et al. (2017) note that the sample size should be related to the effect size where a small effect size warrants a large sample size, and the converse is also true.

Instrumentation is another facet that determines the attainment of internal validity. In the current study, bioelectric impedance is an ideal tool for measuring body composition, but it is underestimated and there is scarcity of information regarding its use (Walter-Kroker, Kroker, Mattiucci-Guehlke, & Glaab, 2011). As a result, there is a probability that the validity of results might have been affected by weak instrument bias (Mohajan, 2017). A weighing scale would have reinforced the findings from the bioelectric impedance. Otherwise the methods of drawing blood for analysis were appropriate. On a different note, there was need to ascertain the reader that the instruments were checked prior to their use to ensure they were operating well to avoid bias.

The study, thereby, might have resulted in type II error which has led to the retention of the hypothesis while in actual sense, the converse might be true if the right research methodology is used (Mohajan, 2017). Errors in measurement that seem to surface in the current study seems to stem from inadequate consideration of the variables under study. All the variables of interest have not been well-defined as well as the best way to measure them. Also, while there are standard stipulations for the sequence of physical activity, it is not apparent whether the physical activity threshold was met or not. The current did not validate the test used; hence, the elimination of instrumentation errors is not guaranteed. Thereby, even for the ELISAs, it was necessary to validate the test based on the measures of interest (Schrijver & Kramps, 1998). Also, the study does not indicate how the allegiance effect was circumvented owing to the fact that the data collection process is missing (Taylor & Asmundson, 2008).

Three subjects were lost as they were discharged from the hospital before the end of the study period. Such loss is considered to result in a negative selection effect because of introduced disproportionality between the study groups (“Experimental Mortality and Internal Validity,” 2012). In the current study, such loss resulted in a change in the number of control groups but the intervention groups remained unchanged. The researchers should have accounted for statistical differences in the disproportionate sizes after the loss to address the element of internal validity. Also, considering that the power of the sample size chosen was not indicated and guaranteed, it is not clear whether a further loss of three subjects might have interfered with the statistical power and associated effect size.

However, other factors of internal validity have not been affected. A clear inclusion/exclusion criteria was present, which is meant to guide the generalization process because the results are specific to a certain population and ascertains the homogeneity of the study population. A homogenous population helps to relate the study results to the selected study population (Salkind, 2010). This criteria is meant to eliminate the selection bias that stems from selecting participants with condition that interfere with the outcomes; thereby, affecting the truth of the results. The element of statistical regression has been addressed by the homogeneity of the two study groups at baseline because the two groups had the same clinical characteristics in reference to the outcome measures of interest (Boston University, 2015). Therefore, both study groups were bound to be affected by the exposure in a similar light. Exposure to the intervention was not bound to be affected by maturation effect in this case because the use of clozapine has not been shown to improve over time based on the condition in question, which is obesity.

Evidently, the results do not have an absolute internal validity. Neither, can they be deemed to have a low validity. The results should be interpreted with caution and more research should be conducted to ascertain the current findings.

Causation

The causal association between exposure and outcomes is not guaranteed based on the fact that there are certain dimensions of the study that could have introduced some bias. Also, the fact that the constituents of the variables under study have not been described and correlated with the outcomes undermines the causal link between exposure and outcomes. It is not certain that the outcomes attained were due to diet control and physical activity because these have not been presented. Establishing a causal link would have sought to find an association between the diet and physical activity as independent variables and nutritional and health status as the dependent variables. Whereas the study is articulate about the manner in which the macronutrients, which have a bearing on the anthropometric measures, have been factored in, the same cannot be said about the metabolic analysis and enzyme immunoassay, which are mainly pegged on the diet quality. The study has not articulately indicated the means used to control the quality of diet and at the end, the authors state, “suppression of appetite for long term continuance among psychiatric patients was difficult, so the dietary restriction was controlled to only 200-300kcal,” implying difficulties in dietary control (Wu et al., 2007). Also, the instrumentation aspect was not addressed well because food diaries, 24-hour recalls, and food frequencies could have been used. The aspect of diet control has not been well addressed. The same goes for the physical activity because a questionnaire to measure this variable would have reinforced the validity of the current results. Whereas there are indications of restricting the diet, subsequent methods to assess the diet quality and affirm the caloric intake of the participants are missing. Thereby, the internal validity of the results might have been jeopardized resulting in the irregularities with the biochemical nutritional profiles that did not match the physical nutritional and health attributes. As far as the study is concerned, medication could have also altered the body’s composition. Hence, more research using valid measurement tools and a clear data collection protocol is needed.

External Validity

Population characteristics are paramount in ascertaining the attainment of external validity. In the current study, it is apparent that the researchers aimed at ensuring that the two study groups were homogenous at baseline. Anthropometric measures between the control and intervention groups were the same at baseline. BMI, waist and hip circumference, body weight, body fat percentage, and waist-to-hip ratio were the same in both the control and intervention groups (Wu et al., 2007). The inclusion/exclusion criteria also affects the external validity by not being too restrictive. In the current study, the inclusion criteria is favorable in promoting the attainment of external validity as it helps to eliminate confounding factors and aid in selecting samples that resemble real-life schizophrenic patients not affected by comorbidities (Patino & Ferreira, 2018).

External validity was affected by the application of dietary guidelines that differed from the typical proportions making up the Taiwanese diet considering that Taiwan was the study setting of interest (Rothwell, 2006). However, health professional in the different spheres helped in assessing the measures used in the study. The study applied dietary guidelines from the American Heart Association as used in a previous study; yet, the Taiwanese have specific average macronutrient requirements. Whereas the subjects were randomized into different groups, there is no clear delineation about the foods used by the different individuals in the two study groups. Thereby, there is a possibility of undermining the eternal validity.

The outcomes were relevant because they were a reflection of the variables of interest. The selected anthropometric measures as mentioned above are all relevant in understanding the nutritional and health status of an obese individual. Also, the blood samples were relevant in understand the nutritional biochemical profile of these obese persons. The measures are congruous to the interventions used because both physical activity and diet control are believed to affect the outcome measures selected for this particular study. However, the measures of diet control and physical activity are lacking.

Thereby, in reference to the current state of external validity, the results cannot be generalized to any specific population. However, it can be replicated while addressing the different issues discussed here to ascertain its reliability in the selected population. The population is well defined, but the subsequent measures used to augment external validity such as administration of the intervention in reference to the diet quality and physical activity have not been accounted.

References

Boston University. (2015). Program Evaluation: Internal and External Validity. Retrieved from https://sphweb.bumc.bu.edu/otlt/mph- modules/ProgramEvaluation/ProgramEvaluation6.html.

Çakmur, H. (2017). Obesity as a Growing Public Health Problem, Adiposity – Epidemiology and Treatment Modalities. IntechOpen: Jan Oxholm Gordeladze. Doi: 10.5772/65718.

Experimental Mortality and Intern al Validity. (2012). Retrieved from https://dissertation.laerd.com/internal-validity-p5.php.

Campbll, D. T., & Stanley, J. C. (1996). Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally and Co.

Experimental Mortality and Internal Validity. (2012). Retrieved from https://dissertation.laerd.com/internal-validity-p5.php.

Guerrera, F., Renaud, S., Tabbò, F., & Filosso, P. L. (2017). How to design a randomized clinical trial: tips and tricks for conduct a successful study in thoracic disease domain. Journal of thoracic disease, 9(8), 2692–2696. Doi:10.21037/jtd.2017.06.147.

Mhaskar, R., Emmanuel, P., Mishra, S., Patel, S., Naik, E., & Kumar, A. (2009). Critical appraisal skills are essential to informed decision-making. Indian journal of sexually transmitted diseases and AIDS, 30(2), 112–119. Doi: 10.4103/0253-7184.62770.

Mohajan, H. K. (2017). Two Criteria for Good Measurements in Research: Validity and Reliability. Annals of Spiru Haret University, 17(3), 58-82.

Open University. (2017). Research objectives. Retrieved from https://www.open.edu/openlearncreate/mod/oucontent/view.php?id=231&section=8.6 .2.

Patino, C. M., & Ferreira, J. C. (2018). Internal and external validity: can you apply research study results to your patients? Jornal brasileiro de pneumologia : publicacao oficial da Sociedade Brasileira de Pneumologia e Tisilogia, 44(3), 183. Doi: 10.1590/S1806- 37562018000000164.

Rothwell, P. M. (2006). Factors That Can Affect the External Validity of Randomised Controlled Trials. PLOS Clin Trial, 1(1), e9. https://doi.org/10.1371/journal.pctr.0010009.

Salkind, N. J. (2010). Inclusion Criteria. Retrieved from https://methods.sagepub.com/reference/encyc-of-research-design/n183.xml.

Schrijver, R. S., & Kramps, J. A. (1998). Critical Factors Affecting the Diagnostic Reliability of Enzyme-Linked Immunosorbent Assays Formats. Rev. sci. tech. Off. Int. Epiz., 17(2), 550-561.

Walter-Kroker, Kroker, Mattiucci-Guehlke, & Glaab, 2011). A Practical Guide to Bioelectrical Impedance Analysis Using the Examples of Chronic Obstructive Pulmonary Disease. Nutrition Journal, 10(35). Retrieved from https://nutritionj.biomedcentral.com/articles/10.1186/1475-2891-10-35.

Wu, M., Wang, C., Bai, Y., Huang, C., & Lee, S. (2007).Outcomes of Obese, Clozapine- Treated Inpatients with Schizophrenia Placed on a Six-Month Diet and Physical Activity Program. Psychiatric Services, 58(4), 544-550.