Lessons and experience sharing on mixed methods research design, from a doctoral study on design coordination in Chinese design institutes – Dr. Rong Zhang

Dear fellows in REC Department,

It is my honor for me to share some of my lessons and experience from the process of doctoral research. You may read widely about research design literature. This one may not as systematic as those ones. However, most of them talk about best practice rather on lessons. Hope my lessons sharing will give you some reminder.

The title of my thesis is “Design Coordination in Design institutes in China: an Information Processing Perspective”. In terms of research design, I applied mixed methods. The lessons and experience of using mixed methods design are what I want to share with you in this article. Mixed methods combine quantitative and qualitative research methods. Some people who are against the use of mix methods argue that quantitative and qualitative research are based on incompatible paradigms assumptions. We have to admit that mix methods have become an increasingly used and accepted approach to conducting social research. As for paradigm assumptions, applying mixed methods commits to a pragmatist position.

The function of mixed methods in my study is mainly for completeness as well as triangulation (or greater validity). Completeness refers to the notion that the researcher can bring together a more comprehensive account of the area of enquiry if both qualitative and qualitative research are employed (Bryman, 2012, p633). Completeness is achieved by combining quantitative and quantitative as following:

  1. a questionnaire survey, as a quantitative method, is used to get a general relationship between interested variables (i.e. organizational structure, interdisciplinary information processing, and design coordination performance and task uncertainty) in design coordination;
  2. semi-structured interviews are used to further investigate the moderating role of task uncertainty on design coordination;
  3. in addition to a static relationship getting from quantitative study, the case study is used to complement the dynamic view that, given task uncertainty, arriving at targeted design coordination performance depended on the organizational structure’s adaptation process, as the fit between task uncertainty and information processing was not static, but a dynamic adaption process (Donaldson 2001; Zhang and Liu, 2015). It is a theory testing process as well.

Triangulation was initially a navigation and land surveying term that used to describe the taking of different bearings to determine one’s precise physical location (Bryman, 2006), in the social sciences, the term triangulation is used to describe the use of multiple data sources to confirm or validate research findings. In my thesis, triangulation is used to provide mutual validation: 1) semi-structured interviews validate questionnaire survey research findings; 2) the case study further validates the research findings from both the questionnaire survey and semi-structured interviews. In a word, the research design in this study is, to a large extent, guided by pragmatic assumptions that often drive research in practice (Bryman 1984; Brannen 2005).

If I had a chance to conduct it once again, I would adjust the sequence of different data collection methods. In my study, research framework was developed based on literature review. Also, research model to test the research framework was developed based on literature review. Based on research model, I developed a questionnaire. If I had a second chance, I would conduct case studies to validate the research framework before piloting questionnaire. In that case, the function of case study would be:

  1. to validate research framework;
  2. to observe the new pattern of research construct (for example, design coordination mechanism is one construct in my research. I’ve identified five different coordination mechanism from literature. If I had conducted case studies before consolidating questionnaire, I would have the chance to observe the coordination mechanism in my research context in China. There would be new coordination mechanisms.);
  3. to test the face validity of the questionnaire. Also, semi-structured interviews would be conducted in the last stage in order to triangulate the results from a questionnaire survey.
Dos and Don’ts while doing empirical study in China

For questionnaire survey, a web-based questionnaire is a good questionnaire distribution tool. For REC students, you are very lucky to have Lime platform, which is enabled by Faculty. Please email the technician if you need an account. The average time for completing the questionnaire should not exceed 15 minutes. If your length exceeds it, a lucky draw with one attractive gift (such as Ipad) and quite a number of souvenir may be helpful to guarantee return rate. For semi-structured interview, some respondent may not allow you to make a record. In that case, you have to make a record by brain and pen.

Measurement of construct

It is highly recommended to use existing measurement scales, as their validity and reliability have been tested. If exiting measurement scales are not suitable for your context, make sure you plan enough time to develop a new one. That is a lesson learnt from my experience with measuring task technology (i.e. task variety and task analysis ability).

Most of the task technology measurement scale was developed and tested in contexts other than building design, Chang and Tien’s (2006) empirical study of subway design being the only exception. Given the importance of the construct and the rarity of empirical studies in building design contexts, testing whether the existing measurement suited the context was deemed worthwhile. After recently-published (i.e., between 2006 and 2014) empirical studies on task technology were reviewed, it was found that most them applied a measurement scale validated by Withey et al. (1983), which includes items on both the individual level (e.g. “To what extent would you say your work is routine”) and the work unit level (e.g. “Basically, unit members perform repetitive activities in doing their jobs”). They argued that their measurement is suitable for work unit level, based on the assumption that the work in a work unit is homogenous in term of task variety and task analysis ability. However, this is not the case in building design discipline teams, where the degree of task homogeneity depends on the division of labor within the team. Firstly thirteen items were generated from literature. Based on piloting study, quite a few modifications were made. After analyzing 40 samples, ten items were consolidated to measure the two constructs. However, based on factor analysis after data collection, it was found only two items could be used to measure task variety. It is difficult to reach a high Cronbach’s alpha index with only two items. The lessons are that a) try to use exiting measuring scale; b) if you want to do an amendment, make sure you conduct enough piloting, collect enough samples to do factor analysis. In the latter case, there will be a contribution for construct measurement. While what I did was neither A nor B. That result in barriers to publish empirical research findings.

Please feel free to contact me by email if you want to discuss more. Welcome you to visit Chongqing.

Wish you all the best in the year of 2018!

Sincerely yours,

Sophia, Rong ZHANG

Author information:

Dr. Rong ZHANG, Ph.D (Primary supervisor: Prof. Anita Liu, Co-supervisor: Dr. Wilson Lu)

Lecturer in Department of Quantity Surveying, School of Construction Management and Real Estate, Chongqing University, P.R.C.

Research interest: Departing from interdisciplinary design coordination in design institutes, my current research interests have extends to the coordination/value co-creation activity between design team and construction team, especially focus on the impact of reward allocation mechanism, ICT tools as well as organizational culture.

Email: Sophia.rongzhang@gmail.com; zhang.rong@cqu.edu.cn

Reference:
  1. Bryman, A. (1984). The debate about quantitative and qualitative research: a question of method or epistemology? British Journal of Sociology, 75-92.
  2. Bryman, A. (2006). Integrating quantitative and qualitative research: how is it done? Qualitative research, 6(1), 97-113.
  3. Bryman, A. (2012). Social research methods: Oxford university press.
  4. Brannen, J. (2005). Mixed methods research: A discussion paper. London, ESRC National Centre for Research Methods, 30.
  5. Chang, A. S., & Tien, C. C. (2006). Quantifying uncertainty and equivocality in engineering projects. Construction Management and Economics, 24(2), 171-184.
  6. Donaldson, L. (2001). The contingency theory of organizations: Sage Publications, Incorporated.
  7. Withey, M., Daft, R. L., & Cooper, W. H. (1983). Measures of Perrow’s work unit technology: An empirical assessment and a new scale. Academy of Management Journal, 26(1), 45-63.
  8. Zhang, R. , &Liu, A.A.M. (2015). Project Organization Structure and Design Coordination in Architecture and Engineering Design Projects: An advanced Structural Contingency Perspective. Proceedings of the 20th International Symposium on the advancement of Construction Management and Real Estate, Hangzhou, China.

A job advertisement:

The school where I currently work in, are recruiting academic staffs from a variety of areas with a competitive salary. Areas include but not limited to construction management, real estate and urban studies. For detailed information, please visit http://url.cn/5BrLOYp