,
Mauricio Cresp Barria2 ,
Valentina Espinoza Zambrano3 ,
Pedro Delgado Floody4 ,
Luis García-Rico5
Introduction. Professional practice is a relevant process for the personal and academic development of students. It uses learning contexts located in real scenarios and enables the acquisition of knowledge, skills, and competencies necessary for professional practice. Objective. To design and validate an instrument designed to evaluate students' satisfaction with theirprofessional practice process. Materials and methods. Mixed-method study, non-probabilistic sample by convenience of 196 participants. The sample is formed by 20 Nutritionists and 176 students belonging to the Nutrition and Dietetics undergraduate program. The statistical software SPSS version 22 was used to analyze reliability by Cronbach's alpha and the Delphi method for content validity. An instrument with 32 items grouped in 4 dimensions was elaborated and applied to 116 students to analyze reliability. The instrument was adjusted to 25 items, and content validity was analyzed by a panel of experts. The judge’sconsensus was the foundation for developing a 29-item version grouped into 4 dimensions. This version was applied to 60 students to obtainCronbach's alpha validation. Results. The instrument presents an excellent level of reliability, with a Cronbach's Alpha coefficient of 0.927. Conclusions. The instrument designed to measure satisfaction with professional practices is a reliable and valid measure, since it allows knowing the students ‘evaluation at the end of their professional practice process. Arch Latinoam Nutr 2023; 73(2): 135-143.
Keywords: Instrument, validity, reliability, undergraduate students.
La práctica profesional es un proceso relevante para el desarrollo personal y académico de los estudiantes, utiliza contextos de aprendizaje situados en escenarios reales, posibilita la adquisición de conocimientos, habilidades y competencias necesarias para el ejercicio profesional. Objetivo. Diseñar y validar un instrumento, que permita evaluar la satisfacción de los estudiantes sobre su proceso de práctica profesional. Materiales y métodos. Estudio de tipo mixto, muestra no probabilística por conveniencia de 196 participantes, distribuidos en 20 Nutricionistas y 176 estudiantes pertenecientes a la carrera de Nutrición y Dietética. Sé utilizó el software estadístico SPSS versión 22, para analizar la confiabilidad por Alfa de Cronbach y el método Delphi para la validez del contenido. Se elaboró un instrumento con 32 ítems agrupados en 4 dimensiones, se aplicó a 116 estudiantes para analizar la confiabilidad. Se ajustó el instrumento a 25 ítems, se analizó la validez del contenido por panel de expertos. El consenso de los jueces dio origen a una versión, con 29 ítems agrupados en 4 dimensiones, se aplicó finalmente a 60 estudiantes para obtener el Alfa de Cronbach. Resultados. El instrumento presenta un nivel excelente de confiabilidad, con un coeficiente de Alfa de Cronbach de 0.927. Conclusiones. El instrumento diseñado para medir la satisfacción de las prácticas profesionales, es una medida confiable y válida, ya que permite conocer la valoración de los estudiantes al finalizar su proceso de práctica profesional. Arch Latinoam Nutr 2023; 73(2): 135-143.
Palabras clave: instrumento, validez, confiabilidad, estudiantes universitarios.
https://doi.org/10.37527/2023.73.2.005
Autor para la correspondencia: Mauricio Cresp-Barría, e-mail: [email protected] / [email protected]
Professional internships at the university are today a personal and professional learning experience for students. For this reason, professional practices represent an adequate source of income to evaluate and know the relevance of university education (1). The relationship with the practice center is the optimal scenario where the skills acquired in professional training are articulated with the tasks, responsibilities and roles required during professional practice (2), promoting significant and lasting learning over time, where the future professionals must incorporate theoretical knowledge, and procedural and attitudinal skills that allow them to successfully meet the needs of society (3).
Training within practical disciplinary processes is of great importance for the personal and academic development of students, since it is considered key at the beginning of their training and in professional life (4).The use of authentic and significant practices during the training process are challenges for the development of skills and aptitudes in accordance with the requirements of the world of work (5), those that contributes to the integral training of the students (2). Using learning contexts located in real settings, it enables the acquisition of knowledge, skills and competencies necessary for professional practice (4) and increases deep learning, autonomy and engagement in their learning process (6). You report international in academic programs of nutrition and dietary do reference to the importance of instruments of evaluation for the professional practices that are valid, reliable with properties psychometric that delete the bias and the subjectivity (7).
There are evaluation experiences for professional practices in nutrition and dietetics, through surveys and validation in evidence-based practical activities (EBP), however, refers to validated instruments that assess knowledge perceived as objective (8). Validate instruments to know the satisfaction of the students with the clinical practices is a potential indicator of an educational program of quality and essential to improve the educational experience (9).
The use of reliable and validated instruments is a guarantor of the quality of the data collected, since a rigorous investigation requires to employ instruments validated with good psychometric properties (10). Of contrary way, the results obtained will not be able to be valid neither generalizable and can cancel any conclusion extracted (11). Designand validate reliable instruments, that obtain information of the context during the professional practices of Nutrition and Dietary program the Universidad Católica de Temuco, will allow feedback on the processes of education and learning in the initial training of professional futures (12), those who prepare to attend the needs of prevention, promotion and nutritional attention of people, families and communities. The present study has by aim, design and validate through the method Delphi and the coefficient Alpha of Cronbach an instrument that allow to evaluate the level of satisfaction of the students of ninth and tenth semester of the formative itinerary, pertaining to the program of Nutrition and Dietetic the Universidad Católica de Temuco, on his process of professional practice.
Delphi was used as a qualitative research method, and Cronbach's Alpha Coefficient was used for the psychometric analysis of the reliability of the data. The expert assessment was carried out using content validity through the Delphi method, which emphasizes anonymous structured communication between people who have experience in a given subject, with the aim of reaching a consensus (13). Developed in the early 1950s by Helmer and Dalkey and incorporated into the health area in the 1970s, it is based on collective intelligence, where a group of selected experts tries to reach a consensus of opinions expressed individually on the subject based on the analysis and reflection of a defined problem (14, 15). This research have been authorized by the Ethics Committee of the Universidad Católica de Temuco, by means of decree of document 13/20, of the month of June of the year 2020. The experts were selected according to the following criteria: a) Doctor in disciplinary areas related to the formative program or to the topic under investigation, b) Academics of Nutritionist training with teaching practice in Chilean Universities and c) Nutritionists practicing in different areas where professional practices are developed. Collecting the opinions of the experts and the collection of the evaluations were carried out during the periods October 2020 - December 2020, and the instrument was delivered personally or sent through Google Drive.
The experts participating in the construction process of the validation instrument were: 1 Doctor in Education (Universidad Politécnica de Madrid, Spain), 1 Doctor in Human Motor Sciences (Universidad Católica de Temuco Chile), 1 Master in nutrition and Food Biotechnology (Universidad Catholica de Temuco). The panel of experts was made up of nutritionists (Table 1), teachers and academics from the following educational institutions: 1 Master in Nutrition, Universidad de Magallanes (Punta Arenas), 1 Master in Nutrition, Universidad de Los Lagos (Osorno campus), 1 Nutritionist, Universidad de La Frontera (Temuco), 1 Doctor, Biologist-Nutritionist and 5 Master in Nutrition, Universidad Católica de Temuco (Temuco), 6 nutritionists belonging to the public sector and 5 nutritionists belonging to the private sector.
The evaluation of the criteria of sufficiency (items belonging to the same dimension), clarity (the item is easily understood, that is, its syntax and semantics are adequate), coherence (the item has a logical relationship with the dimension or indicator it is measuring) and relevance (the item is essential or important, that is, it should be included) was considered using a Likert scale, and the level of compliance of the items was indicated: Does not meet the criterion (1), Low level (2), Moderate level (3) and High level (4). In addition, each dimension had an open field for observations to incorporate comments and suggestions related to the items (Table 2).
The exhaustive reviews of the contributions of the experts, together with the analysis of the structure of theinstrument, madeitpossibletoreviewtheadequacy and understanding of the items that comprise it (15). Considering the evaluations, observations, and consensus of the experts, he developed an instrument that includes 29 items. For the date collection period and to analyze the reliability of the results in its final version, it was physically applied to 60 students (48 women and 12 men), during the months of January to March 2021, whose professional practices had completed in the areas of professional training. The evaluation of the 29 items, grouped into 4 dimensions: practice center (6 items), academic training with application to the labor field (9 items), about the teaching clinic (7 items) and about the clinical supervisor (7 items); It was carried out using the Likert scale assessment: Never (1), Almost never (2), Sometimes (3), Almost always (4), Always (5).
A review of the literature was made to determine the dimensions related to design and construction. In the case of training instances in practice centers or clinical experiences, it is determined that they are of great importance for the care of disciplinary and generic competencies that represent the basis for professional practice (19). In the academic field, in the case of Chile, program accreditation is the main regulatory instrument for the quality of undergraduate education (20, 21). Being the operating conditions one of the most important dimensions to evaluate in the academic units, emphasizing the teaching staff with criteria related to the creation and research of the teaching staff, academic level and experience, for the generation and application of knowledge in the student training (22). Regarding the role of the supervisor, it is worth mentioning that he is the manager to facilitate the teaching-learning process in an environment different from that of the classroom (19), since he not only instructs, but also encourages observation, analysis, investigate and make decisions. In this way, promote the progress of skills and attitudes that lead to the acquisition of technical knowledge, whose ultimate goal is to be able to act for the benefit of society (23). Another important actor is the nutrition teaching clinician, whose role is to ensure the student's exercise. In other words, the tutor has to support and advise students in their adaptation, development and training process, enhancing their abilities and helping to overcome the difficulties that appear throughout the educational process (24,25).
Once the contributions were collected and included, an instrument called "Satisfaction survey of professional practices in the Nutrition and Dietetics program " was designed and validated, whose objective is to analyze the perception of the student, on the assessment that he gives to the process of practical professional of the program of Nutrition and Dietetics. The instrument includes the following structure: title, objective, explanatory table for the qualification of the items and 29 items grouped into 4 dimensions: practice center, academic training with application to the labor field, about the teaching clinic and about the clinical supervisor.
The non-probabilistic sample, by convenience, was composed of 176 university students (83% women and 17% men) belonging to the ninth and tenth semesters of the Nutrition and Dietetics program who studied the professional practice in the areas of clinical nutrition, primary care, collective nutrition and health promotion, during the years 2019-2020. In addition, a panel of 20 experts was formed, equally distributed between Nutritionists with teaching experience (minimum 3 years) and Nutritionists with performance in various areas of the professional field. The distribution of the sample is shown in the following table.
For the psychometric study of the reliability analysis, the statistical software SPSS version 22 (SPSSTM Inc., Chicago, IL) was used to determine the Cronbach's Alpha coefficient (16). This type of analysis is widely used for the validation of research instruments and allows the internal consistency of an instrument, through the degree of correlation between its items (17). The interpretation of Cronbach's Alpha coefficient establishes the following: Invalid reliability (0.00 to 0.53); Low reliability (0.54 to 0.59); Reliable (0.60 to 0.65); Very reliable (0.66 to 0.71); Excellent Reliability (0.72 to 0.99) and Perfect Source Reliability (1.00) (18).
Table n°3 presents the internal consistency (reliability per dimension) of the items of the instrument in its first version (32 items). A high reliability coefficient was obtained for the whole instrument = 0.949 and the results per item were also satisfactory. Each dimension obtained reliable results: dimension 1 α= 0.891; dimension 2 α= 0.957; dimension 3 α= 0.920and dimension 4 α= 0.925.
Table 4 presents the internal consistency of the instrument, by dimension and item in its final version (29 items). Each dimension obtained reliable results: dimension 1 α= 0.854; dimension 2 α= 0.889; dimension 3 α= 0.957 and dimension 4 α= 0.947. The reliability analysis of the instrument yielded a Cronbach's alpha coefficient of 0.927, which meant that the instrument is reliable.
The objective of the present investigation was to design and validate an instrument, which allows to evaluate the satisfaction of the students about their professional practice process. The foregoing considering that higher education institutions should be guarantors of the quality of professional training and of health in the face of society (26). In this way, have a valid and reliable instrument that evaluates the dimensions of the practice centers, the academic professional role and the clinical supervisor in the areas of professional training; primary care, clinical nutrition, collective feeding and health promotion, taking into account the perspective of teachers and students, which is endorsed by the support of the triad of clinical learning (27). The validation of the content by the experts, who through an evaluative test (Delphi method) that considered the sufficiency, clarity, coherence and importance of the elements in the four dimensions, allowed adjustments to be made in the number of items, vocabulary, grammar and editorial, thus obtaining the final version of the instrument. Regarding the implications for health professionals, it is worth noting the unavoidable need to disseminate guidelines for the preparation of questionnaires, their consolidation and adaptation through content validation through expert judgment (28), not being exempt from this situation academic programs of nutrition and dietary in Chile. International studies recommend incorporating in the validation both experts linked to the field of research by subjects of the methodological aspects of the instrument, as well as comprehensive potentialities of the target population (29), a situation that for the present investigation is protected with N = 20. experts and N = 166 members of the target population.
In this sense, national studies of design and validation in health areas referring to the performance of the teaching clinic, have reported a high value of Alpha of 0.970 of its four factors of the scale (personal qualities, skills in education, skills under evaluation, favorable clinical environment) (26). Similar results to the present study, which presents consistency values for the educational clinical item with an Alpha of 0.957. Focused national investigations studied the psychometric properties to assess perception, satisfaction, and learning results in graduate students in Health Sciences, which presented internal consistency levels of 0.963 (30). In turn, for the present study, the reliability through Cronbach's Alpha coefficient, showed an internal consistency of the instrument for each one of the dimensions, granting an excellent reliability (0.927), which allows verifying that the designed instrument is valid of content that is, the instrument measures what it intends to measure (31).
Considering the slight increase in Cronbach's Alpha coefficient (0.931) when removing the first item of the second dimension, which is related to the duration of professional practice (number of hours), the researchers decide to keep the item, since it assesses the number of hours that the student remains in the institution and/or company where they carry out their professional practice, so that the instrument is not modified in its final version. This instrument, designed and validated by expert judgment and psychometric analysis, aims to be a contribution in any of the stages of the training itinerary in which the students are. In this way, demonstrating and systematizing the teaching and learning processes in various contexts is of the utmost importance. Especially from the discipline of nutrition and dietetics, as in the development of professional practices.
The use of validated and reliable instruments for the collection of information in the practice area of professional training of the Nutrition and Dietetics program at the Universidad Católica de Temuco is considered strength by the researchers. One limitation to consider is the n= 60 of the sample of the last application, projecting for future research n= 80 or higher for each academic year. Within the strength, it is considered an advance to know the satisfaction of the students regarding the infrastructure of the practice center or clinical field where they carry out their professional practice and the perceptions in the relationship established with the teaching supervisor and teaching clinician, who finally evaluate and describe the students' exercise.
Content validity can often be subjective, but in the case of this study, objectivity could be added through the use of statistical processes, which provide validity and reliability, so that the rigor in the construction of the instrument as well as the agreement between experts is the best. way to correctly validate an instrument.The various stages of content and reliability validation, added to the results, show that the instrument has an excellent level of reliability, with a Cronbach's Alpha coefficient of 0.927. Therefore, the instrument designed in its final version with 29 items grouped into 4 dimensions, is valid and reliable, it is also simple and allows to assess the practice center, the academic training linked to the practice and the role of those who accompany the practice student process. The instrument, consequently, allows to know the assessment given by the ninth and tenth semester students at the end of their professional practice and allows having an instrument that can be used in the Nutrition and Dietetics program and others related to Health Sciences. It should be noted that the instrument present in the academic field can be used and modified in areas for Health purposes, such as the Social Sciences and the careers of the Faculties of Education that also have professional practices in their training itineraries. It is recommended that in order to replicate the instrument in other areas, it will always be necessary to use expert judgment, being optimal to articulate it with statistical treatments so as not to lose the objectivity of the study area.
The authors thank the Nutritionists who participated as judges in the validation of the content and the students of the Nutrition and Dietetics program of the Universidad Católica de Temucoin the reliability analysis.
The authors declared no conflicts of interest.
Recibido: 12/04/2023
Aceptado: 26/06/2023