Jornada Curso Juan Marin · 2010-05-17 · 17/05/2010 1 10.DOE-CUALITA 1ª Jornada 2010....

17/05/2010

1

10.DOE-CUALITA 1ª Jornada 2010. Metodología de Investigación del DOE - Metodología de análisis

cualitativo

G d i ti ió

JornadaCurso

Juan Marin

Universidad Politécnica de Valencia

Grupo de investigación enReingeniería, Organización,trabajo en Grupo, Logística y Excelencia

ROGLE

Objetivo

La jornada se centra en la Grounded Theory (teoría fundamentada) enmarcándola dentro del contexto de metodologías de análisis cualitativo. El enfoque principal es dar a conocer al alumno cómo puede poner en práctica esta metodología para aplicarla enponer en práctica esta metodología para aplicarla en sus investigaciones.

Al terminar el curso el alumno podrá: 1.- Enumerar y diferenciar los principales métodos

cualitativos relacionados con la investigación en organización de empresas.

2.- Conocer con detalle en qué consiste la Grounded Theory(teoría fundamentada) sus posibles aplicaciones, sus ventajas y sus inconvenientes

ROGLE

Juan Marí[email protected]

RengineeringOperationsGroupWorkLogisticsExcellence

ventajas y sus inconvenientes. 3.- Experimentar los pasos necesarios para aplicar la

Grounded Theory 4.- Saber qué software existe para dar soporte al uso de

esta metodología 5.- Ver ejemplos de uso de ATLAS-Ti

17/05/2010

2

Cómo nos vamos a organizar

Parte 1 Diferenciar cualitativo de cuantitativo

Dinámica Teoría

Qué preguntas son adecuadas en investigación cualitativa Qué preguntas son adecuadas en investigación cualitativa Dinámica Teoría

Métodos cualitativos Tormenta de ideas Teoría

Descanso Parte 2

ROGLE



Detalle de la Grounded Theory Descanso Parte 3

Aplicación práctica de Atlas-ti Como soporte para la Grounded Theory Para otras funciones/metodologías

Materiales de apoyo

Todo está accesible en https://poliformat.upv.es/access/content/group/E

SP_0_87/SubdInvestigacion/Jorn_DOE2010 Es un alias a esta ruta: Es un alias a esta ruta:

https://poliformat.upv.es/portal/site/OCW_22388_2010

ROGLE



17/05/2010

3

Visión general de las metodologías cualitativas

ROGLE



PARTE 1

Actividad por parejas

En grupos de 2-3 personas: Describe las principales características

que diferencian la investigaciónq gcualitativa y la cuantitativa

ROGLE



17/05/2010

4

Aclarando Cualitativo vs Cuantitativo

Quantitative emphasizes 1. Starting with specific

hypotheses or questions derived from theory/previous research

2. Selecting a large, random

Qualitative emphasizes 1.Starting with general research

problems and not formulating hypotheses (although hypotheses may emerge from the data analysis and may shift during the course of the study).

http://www.cedu.niu.edu/~sorensen/502/powerpoint/topicD/qlnotes.htm

2. Selecting a large, random sample representative of the population

3. Using objective instruments (e.g. achievement tests, attitude scales, etc.)

4.Presenting results using statistics and making inferences to the population.

2. Selecting a small, purposive sample (not random) which may or may not be representative of the larger population.

3. Using relatively unstructured instruments (e.g. interviews, observations, etc.) and “intense” data collection (e.g. over extended periods of time).

4. Presenting results mainly or exclusively in words and de-emphasizing generalizations to the

ROGLE



p p

5. “Distance” between researcher and subjects and emphasis on following the research plan

p g gpopulation

5. Researcher awareness of their own orientations/biases/experiences and personal interaction in the context with an emphasis on flexibility in the research

¿coincide con lo vuestro? ¿Qué falta o qué sobra a esta lista?

Enfoque cualitativo vs cuantitativo para el análisis de datos

Differences in qualitative and quantitative interviews Quantitative interviews are similar to “survey research” in that

there are mainly fixed-choice questions and generally random samples.

Interviews in qualitative studies are generally open-ended and generally use small purposively selected samples


generally use small, purposively selected samples. Differences in qualitative and quantitative

observations Quantitative observations use checklists with preliminary work

done prior to the observation. Tally marks are generally used to identify behaviors.

Qualitative observations use words to describe behaviors and attempt to describe what is happening in a context.

Differences in qualitative and quantitative contentROGLE



Differences in qualitative and quantitative content analysis Quantitative content analysis uses counting in terms of researcher

identified categories. The categories are sufficiently precise to enable different coders to arrive at the same results.

Qualitative content analysis develops categories after reviewing the content in an attempt to understand participants’ categories.

17/05/2010

5

Ahora que ya sabes las diferencias entre CTT y CLV

En grupos de 2 o 3. ¿para qué tipo de preguntas de

investigación usarías la investigacióng gcualitativa?

ROGLE



Para qué es adecuada la investigación cualitativa

What is happening in a specific setting? What are the meanings and perspectives of participants in the

setting? What do specific events mean to the people involved? What are the relationships regarding the structure, occurrence,

and distribution of events over time?


Are events organized into patterns and principles that guide everyday life?

How are events in a particular setting related to events at other system levels inside and outside the setting?

How does the manner in which life in this setting is organized compare with other ways of organizing life in other places/times?

Qualitative research is useful for describing or answering questions about particular, localized occurrences or contexts and the perspectives of particular groups toward events, beliefs practices It is also useful for exploring a complex

ROGLE



beliefs, practices. It is also useful for exploring a complex research area about which little is known or for beginning to understand a group or phenomenon. Can provide the specific, concrete details to guide an

understanding of a particular setting Can provide interpretation of local meanings that activities and

practices have for the group engaged in them and illuminate differences across settings.

17/05/2010

6

É

Vamos a ir repasando uno a uno y vostros me preguntáis lasdudas que os surjan (yo os reponderé lo que sepa ;-) )

ROGLE



MÉTODOS CUALITATIVOS Y SU RELACIÓN CON LA ORG. DE EMPRESAS

Tormenta de ideas

¿Qué métodos cláramentecualitativos conocéis?1. Casos1. Casos2. Entrevistas3. Grupos de discusión4. Preguntas abiertas5. Analisis de contenido

óROGLE



6. Observación participante7. Trabajo en equipo….

17/05/2010

7

Esquema general de métodos en función del tipo de recogida de datos

• Semiótica• Análisis de discurso• Análisis de contenido

• Revisiones bibliográficas• Análisis histórico

Docuementosno “forzados”

• Análisis histórico

• Historias de vida• Entrevistas en profundidad• Entrevistas enfocadas• Entrevistas de grupo cara a cara• Delphy (entrevista grupo No interactivo)

Entrevistas

ROGLE



• Estudios etnográficos• Casos

• Observación participante• Action research

Observación

Semiotics/Discourse Analysis

This is the study of linguistic units. It looks at the relationship between words and

their meanings stressing the system of relations between words as a source of meaning.

Views language as a social construction. Words are viewed as signs that bring together a

concept and an image, derive their meaning from their place within an articulated system, are arbitrary, can be put together in combinations/patterns (syntagmatic relationships), and their use excludes other choices (express

ROGLE



and their use excludes other choices (express paradigmatic oppositions).

Questions focus on verbal interaction and dialogue.

Primary data collection strategy is recorded dialogue (text-based, or audio/video recorded).

17/05/2010

8

Semiotics/Análisis de contenido

Documents and artifacts can include materials produced by the subject (writing portfolios), personal documents (diaries, letters), records (memoranda, meeting minutes, newsletters), ( e o a da, eet g utes, e s ette s),memorabilia (yearbooks, newspapers), documents/photographs from historical archives.

Includes analysis of written records such as textbooks, novels, newspapers, music, speech transcripts, photographs, birth and death certificates, curricula, diaries, letters and well as

ROGLE



, , ,analysis of non-written records such as audio tapes, video tapes, computer files.

Semiotics/Historical analysis

Data for historical studies include newspapers, legislative documents, court testimony, diaries, committee meeting records, yearbooks, memos, relics, photographs, reports, and sometimes interviews with living reporters of past events.

These are categorized into primary and secondary sources. Primary sources are first-hand information sources - -

eyewitness reports and original documents. Secondary sources include second-hand information - a

textbook author’s description of a theory, someone who heard about the event but did not experience it firsthand.

A major problem with much historical research is over-reliance on secondary sources.

ROGLE



reliance on secondary sources. Historical data should not be accepted at face value.

Diaries, memoirs, reports, testimonies reflect the interpretation of the author and may be biased. Sources should be examined for authenticity and

truthfulness using external and internal criticism.

17/05/2010

9

Interviews (deep/Group/focused)

Interviewing is used to gather information in the subjects own words from which insights on their interpretations can be obtained. Qualitative researchers generally use open-endedQualitative researchers generally use open ended

informal interview techniques (and generally do NOT use fixed-response questionnaires or surveys).

Subjects are encouraged to talk about experiences, feelings, beliefs.

The most important characteristic of a good qualitative interviewer is to be a good listener (i.e. don’t talk!).

Sometimes structured interviews are conducted toROGLE



Sometimes structured interviews are conducted to compare views across different subjects, groups, or settings.

Interviews also vary in that there may be one-time interviews, multiple interviews with the same participant, multiple interviews with various participants, or group interviews.

Entrevistas/Delphi

En el libro de E. De miguel (1993) tenéis unadescripción del método

Otra referencia donde lo explica de maravilla: Day J and Bobeva M (2005) “A Generic Toolkit for the Day J and Bobeva M (2005) A Generic Toolkit for the

Successful Management of Delphi Studies” The Electronic Journal of Business Research Methodology Volume 3 Issue 2, pp 103-116, available online at www.ejbrm.com

ROGLE



17/05/2010

10

Historias de vida/autobiagrafías

This method seeks to report on and document the history of a person’s life. These can be objective, historical, artistic, narrative, personal, collective, institutional. Many researchers study problems st tut o a a y esea c e s study p ob e sanchored in their personal biographies.

ROGLE



Estudios de casos (etnography)

In a case study, one case is selected of particular interest. This case is observed on a regular basis over a period of

time. The focus of the research is on process. The question is focused on what can be learned from this The question is focused on what can be learned from this

particular case. The case could be one individual, or one organization, or one

group, etc. An intrinsic case study is undertaken because the researcher

wants a better understanding of a particular case, not because it represents other cases or illustrates a particular trait.

Instrumental case studies involve examination of a case inROGLE



Instrumental case studies involve examination of a case in order to provide insight into an issue or refine a theory.

Collective case studies involve study of a number of cases in order to understand a broader phenomenon or a general condition.

Uses multiple data sources.

17/05/2010

11

Participative Inquiry/Action Research

This method attempts to make qualitative research more humanistic, holistic, and relevant to the lives of humans.

It views participants as co-creating their reality through participation experience and action


through participation, experience, and action. There are four phases of action research:

the co-researchers agree on an area of inquiry ideas and procedures are applied in everyday work/life. co-researchers become fully immersed in the

activity/experience co-researchers reconsider the original research problem.

The goal is to produce knowledge and action directlyROGLE



The goal is to produce knowledge and action directly useful to a group of people.

A second aim is to empower people through the process of constructing their own knowledge.

Methods include unstructured observations, journaling, surveying, and reviewing documents/records.

Teoría fundamentada

One of the most widely used interpretive strategies used in qualitative research. Rooted in sociology. Pero en OE es prácticamente desconocida.

The focus in grounded theory is to unravel elements of experience and use interrelationships to build theoryexperience and use interrelationships to build theory to understand the nature and meaning of an experience for a

particular group of people in a particular setting. Researcher constantly reviews field notes, etc. and attempts

to provide explanations to guide future observations. These observations then confirm or disconfirm the

explanation. There is a constant shift from observing and collecting data

to analyzing. ROGLE



The researcher stresses open processes in conducting the research rather than fixed methods and procedures.

Focus is on process questions - experience over time or change that may include stages or phases.

Primary data collection strategy is recorded interviews. Other data sources may include observation, journaling, diary, memos.

17/05/2010

12



Dinámica Teoría



Descanso Parte 2

ROGLE





ROGLE



DESCANSO

17/05/2010

13

Grounded Theory

ROGLE



PARTE 2

El proceso de la Grounded TheorySALDANA, J. (2009). The Coding Manual for Qualitative Researchers.Sage Publications .

audi

o… v

ideo

, fot

os, a

ROGLE



Text

o,

17/05/2010

14

Recoger datos Preparar datos

Memos

AnalisisDatos

p Analizar datos Construir teoría Escribir memos (bitácora de todo el proceso:

pensmientos, intuiciones, sesgos, problemas, decisiones…)

GT

ROGLE



PASOS

Preparar datos: Recomendaciones para transcripcionPD provenientes de entrevista

academic transcriptions must be based on a system of rules: protocolo

Recorded word for word, exactly as said, including any nonverbal or background sounds (e.g., laughter, sighs, coughs,claps, snaps fingers, pen clicking, and car horn).

http://www.audiotranskription.de/english/transcription/how-to-transcribe/tipps.html

McLelland, 2003 Field Methods, Vol. 15, No. 1, February 2003 g , p g, )February 2003 63–84

ROGLE



17/05/2010

15

Analizar datos

Data analysis in qualitative studies is inductive. The researcher looks for patterns that emerge from the data. There are no pre-defined variables to focus on for analysis. The variables are identified through review of the data. There are no agreed upon approaches to analyzing qualitative data. Analysis of qualitative data is a difficult task


Analysis of qualitative data is a difficult task. Generally you have piles of field notes, piles of transcripts, piles of documents -

- all of which you must sort through and determine what is important. Unlike quantitative data which are analyzed in routine ways, you must find your

own idiosyncratic method of analysis. You must systematically

(1) search - becoming familiar with the data and identifying main themes (2) describe - examining the data in depth to provide detailed descriptions of the

setting, participants, activities, etc. (3) classify - categorize and code data by physically grouping data into themes (4) integrate and interpret - synthesize the organized data into general

conclusions or understandings. The p ocess is length and time cons ming

ROGLE



The process is lengthy and time consuming. The process is iterative - - the researcher will go through these stages more

than once. A nd the process tends to lose structure and become unpredictable - i.e.

interpretation may lead to classifying, describing may lead to interpretation, etc.

Organizar datos

Qualitative data consist primarily of verbal descriptions which need to be analyzed for themes and patterns which the researcher then describes and illustrates with examples that include quotes and excerpts.

The process of data collection and analysis occurs simultaneouslysimultaneously.

The first step is to determine a system for managing the data. Make sure the data (field notes, transcripts, observer comments,

memos, reflections) are dated, organized, and sequenced. You can also organize computer files and create separate folders

for different types of data and stages of analysis. Copy all data so that you can mark up or underline important parts, cross out unneeded parts, even literally cut sections out without losing your original data. M k b k fil llROGLE



Make computer backup files as well. Computerized processing - there are some programs

available for coding, searching and sorting qualitative date (Ethnograph, Nudist, ATLAS-ti, Max QDA, etc.).

Other technologies can also assist in qualitative research word processing programs can help manage field notes,

spreadsheets for tracking participants and setting characteristics, graphics packages for creating diagrams, etc.

17/05/2010

16

Leer y Escribir notas, “memos”

The first step after data are organized is to read the data to get a sense of it.

This means reading everything - including comments in the margins memos etccomments in the margins, memos, etc.

As you read, make additional notes in the margins, underline sections or issues that seem important.

Document themes or common threads that reoccur.G ll t thi t ( th )ROGLE



Generally you repeat this step (more than once) before moving to the next step.

Codificar Qualitative analysis is a process of breaking data into smaller units,

determining the import of these units, and putting them back together in an interpreted form.

Breaking down the data is a process of classifying or coding. category is a classification of ideas or concepts Lower-level categories can be organized into higher and more abstract conceptual

categories. The categories one researcher uses to organize qualitative data may not The categories one researcher uses to organize qualitative data may not be the same categories another researcher would use to organize the same data.

Different authors recommend different ways of defining categories. participant acts, activities, participant meanings, relationships among participants,

settings, perspectives of participants, participants ways of thinking, regularly occurring activities, infrequently occurring activities, methods, group acts, individual acts, etc

Each of these categories may be further broken down into sub-categories. Categories should always be provisional and the researcher should not become too

attached to initial categories. Begin qualitative analysis by coding - review transcripts and documents

ROGLE



Begin qualitative analysis by coding - review transcripts and documents and identify categories as they emerge.

Then organize by category - - categories result from reading and re-reading field notes, etc. and noting regularities related to ideas, activities, setting, structures, etc. Data are sorted into the categories and resorted and re-categorized as additional

data are collected and analyzed. Categories are then merged into patterns. Patterns are links among

categories that further integrate the data and are used as the basis for reporting.

17/05/2010

17

Analizar Analysis of qualitative data involves organization, classification, categorization, a search for

patterns, and synthesis. Analysis is recursive. There are no guidelines for determining how much data and analysis are needed to support

conclusions. Depends on the nature of the study, the amount of data, and the analytic abilities of the researcher. But typically, it takes longer than analysis for quantitative studies.

There are three key strategies used for qualitative analysis:y g q y Constant -comparison - comparing identified topics and concepts to determine distinctive characteristics

so they can be placed in the appropriate categories. As a new topic or idea is identified, it is compared to the existing categories. The researcher asks “Is this topic or concept similar to or different from existing categories?” Categories are then added or modified to fit the new data then tested with additional data. This method is similar to the analytic approach except that data are collected from several cases before

initial hypothesis development. Data are examined for categories, patterns, consistencies and inconsistencies. Steps in this approach include:

collect data from several cases, identify important issues or recurring events and use them to create categories, collect additional data to provide more examples for each category and to elaborate on dimensions within the categories, describe how the categories account for documented events then reformulate some categories and delete others as data dictate, identify patterns and relationships, develop a theory by continuing to collect and compare data and refining categories and relationships.

Negative case and discrepant data analysis - searching for data that are negative or discrepant from the main data. Information that contradicts an emerging category or pattern is a negative case. to examine counter evidence.

ROGLE



e de ce This method provides a counterbalance to the tendency to stick with first impressions.

Analytic Inductive Analysis - This method is suggested if the researcher has a specific problem, question or issue in mind and wants to develop a theory or hypothesis about it.

There are 7 steps : define the event of study, develop a tentative hypothesis, evaluate another case to see if it supports the hypothesis,

reconsider and restate the hypothesis, evaluate cases that may disprove the hypothesis, reformulate the hypothesis based on these observations, continue until the hypothesis is confirmed.

Data are analyzed for corroboration of ideas or themes. Use of various methods of triangulation to support findings. Triangulation could include information

from various sources, various participants, various settings, various observers, etc.

Interpretar Interpreting is the reflective, integrative and explanatory part of dealing with the

data. Interpretation is personal. There are no hard and fast rules for interpreting meaning from

qualitative data. It is dependent upon the perspective and abilities of the researcher. The researcher must explain the meaning of the data to others. Interpretation requires more conceptual and integrative thinking than data analysis

and involves abstracting important understandings from the detailed and complex data. Th f i h i i i h d h i i i d h b The focus is on what is important in the data, why is it important, and what can be learned from it.

Some general strategies can help in data interpretation. Pay attention to your research focus, it should guide the researcher in selecting portions of

data for interpretation. Look at categories that contain large amounts of data as they are likely to identify important

concepts or practices for interpretation and are likely to contain links or sequences. Focus on the patterns, what do the interrelationships between categories suggest? Examine existing studies related to your topic to help in identifying interpretations. Talk with peers about the data and discuss interpretations. Step back from the data every now and then and take time to reflect.

Note, while analysis and interpretation of qualitative data rely heavily on the ROGLE



, y p q y ypreferences and abilities of the researcher, qualitative researchers do not have carte blanche to rely strictly on feelings or preferences. If qualitative research were based solely on producing unsubstantiated opinions which

ignored contradictory data and biases, it would be of little value. Six questions can help check data quality:

Are data based on actual observations or hearsay? Is there corroboration by others? In what circumstances was the observation made? How reliable are those providing data? What motivations might have influenced a participant’s report? What biases might have influenced how an observation was made or reported?

17/05/2010

18

Cómo nos puede ayudar Atlas-Ti

Atlas-ti es un soft para dar soporte y archivo a todos los pasos necesarios en la GT Transcripción Visor de PD Codigos, super-códigos, relaciones, mapas Codificación y Recuperación de citas Codificación y Recuperación de citas Memorandos y registro de ideas Compartir el trabajo entre varios investigadores …

Atlas-ti es un gestor de fragmentos y etiquetas Permite etiquetar fragmentos y recuperar esos fragmentos de muy

diverso modo. Por ejemplo… Listar códigos sobre un tema y agruparlos en categorías Codificación de preguntas abiertas de cuantitativo

Para identificar los códigos y documentar los pasos seguidosROGLE



Para identificar los códigos y documentar los pasos seguidos Meta análisis o revisiones de bibliografía Para exportar los códigos para analizarlos después (necesitas que los

PD sean independientes) Análisis de contenido

Fichas de lectura Exploración cualitativa de un fenómeno (sin llegar a una construcción

teórica posterior)



Dinámica Teoría



Descanso Parte 2

ROGLE





17/05/2010

19

ROGLE



DESCANSO

Manual vs Atlas-ti

ROGLE



PARTE 3- EJEMPLOSPRÁCTICOS

17/05/2010

20

Ejercicio por parejas

Analizamos las expectativas del grupo Son 3 preguntas abiertas

Buscamos resumir las expectativs, usos y problemas quetienen las personas que asisten al taller. Identificando: Clasificación de los usos, problemas y expectativas Relación entre problemas y expectativas Relación entre expectativas y contenidos reales del taller.

ROGLE



Mis resultados con Atlas-ti

Qué es a lo que he llegado

Mostrar algunosdetalles de cómo lo hedetalles de cómo lo he hecho.

ROGLE



17/05/2010

21

Usar ATLAS-Ti como codificación para cuantitativo

Objetivos Analizar la frecuencia de aparición de estudios por

Tipo de investigación Área geográfica Foco

Referencias Referencias Ahire, S. L. (2001). Linking operations management students directly to the real

world. Interfaces, 31(5), 104-117. Albritton, M. D., McMullen, P. R., & Gardiner, L. R. (2003). OR/MS content and

visibility in AACSB-accredited US business programs. Interfaces, 33(5), 83-89. Alfalla-Luque, R., & Machuca, J. A. D. (2003). An empirical study of POM

teaching in spanish universities (II) - faculty profile, teaching and assessmentmethods. International Journal of Operations & Production Management, 23(3-4), 375-400. doi:10.1108/01443570310467311

Alfalla-Luque, R., & Machuca, J. A. D. (2000). El profesorado de dirección y gestión de producción / operaciones en la universidad española: Un estudio empírico. Cuadernos De Economía y Dirección De La Empresa, (6), 285-316.

Alptekin, S. E., DeTurris, D., Macy, D. J., & Ervin, J. E. (2005). Development of aROGLE



Alptekin, S. E., DeTurris, D., Macy, D. J., & Ervin, J. E. (2005). Development of a flying eye: A project-based learning experience. Journal of ManufacturingSystems, 24(3), 226-236.

Andersen, B., Brandolese, A., Capello, A., Doumeingts, G., Garetti, M., Mertins, K., et al. (1996). Engineering education on production management. differentlevels and degrees in five european countries: A comparison. Production Planning& Control, 7(5), 529-539

Heineke, J., & Davis, M. M. (2007). The emergence of service operationsmanagement as an academic discipline. Journal of Operations Management, 25(2), 364-374. doi:10.1016/j.jom.2006.11.003

…

Mis resultados con Atlas-ti

Qué es a lo que he llegado Mostrar algunos detalles de cómo lo he hecho.

ROGLE



17/05/2010

24

Resumen

Si las categorías/etiquetas las vas a crear manualmente, creo que una de las mejores opciones es Atlas-ti -si sigues la metodología Grounded Theory-. Hay otros (N-Vivo, QDA, G ou ded eo y ay ot os ( o, Q ,WEFT, incluso hay alguno via web: online-QDA http://onlineqda.hud.ac.uk/) pero a mi me gustan menos (sueles encontralos como CAQDAS en google, o Wikipedia). Casi todos son de pago.

Si lo que quieres es una programa que saque las ROGLE



q q p g q qcategorías automáticamente, la cosa está más complicada, suelen ser caros y además solo son fiables en el entorno para el que se crearon (por ejemplo enseñanza de idiomas, mapas conceptuales o similares).

How –to Referencias en la web

Generados por mi http://polimedia.upv.es/polimedia.v2/lanza/index.asp?id

=ceb62aae-f068-114f-8751-68e5541953ef&existe400=1 http://polimedia.upv.es/polimedia.v2/lanza/index.asp?id

=01f0536c-7ed0-4c48-a147-41fd32a5c1f8&existe400=1

Generados por Atlas-ti http://www.atlasti.com/video/article/coding-

basics.html?cHash=644d875ed1 …

Otros..ROGLE



http://www.youtube.com/watch?v=RbzVVcPlsU8 http://www.youtube.com/watch?v=beJlPA-5iB0

17/05/2010

25

Dudas o comentarios

ROGLE



Fin

ROGLE



Jornada Curso Juan Marin · 2010-05-17 · 17/05/2010 1 10.DOE-CUALITA 1ª Jornada 2010....

Documents

Transcript of Jornada Curso Juan Marin · 2010-05-17 · 17/05/2010 1 10.DOE-CUALITA 1ª Jornada 2010....