[1] V. R. Basili, F. Shull, and F. Lanubile, "Building
Knowledge Through Families of Experiments," IEEE
Transactions on Software Engineering, vol. 25, no. 4, pp.
456 - 473, 1999.
[2] K. Beck, Test Driven Development -- by Example.
Boston: Addison Wesley, 2003.
[3] T. Bhat and N. Nagappan, "Evaluating the efficacy of
test-driven development: industrial case studies," in
ACM/IEEE international symposium on International
symposium on empirical software engineering, Rio de
Janeiro, Brazil, 2006, pp. 356 - 363
[4] D. T. Campbell and J. C. Stanley, Experimental and
Quasi-Experimental Design for Research. Boston: Houghton
Mifflin Co., 1963.
[5] G. Canfora, A. Cimitile, F. Garcia, M. Piattini, and C.
A. Visaggio, "Evaluating Advantages of Test Driven
Development: a Controlled Experiment with
Professionals," in International Symposium on Empirical
Software Engineering (ISESE) 2006, Rio de Jaiero, Brazil,
2006, pp. 364-371.
[6] N. E. Fenton and S. L. Pfleeger, Software Metrics: A
Rigorous and Practical Approach: Brooks/Cole, 1998.
[7] D. Gelperin and W. Hetzel, "Software Quality
Engineering," in Fourth International Conference on
Software Testing, Washington, DC, June 1987.
[8] B. George, "Analysis and Quantification of Test
Driven Development Approach MS Thesis," in Computer
Science Raleigh, NC: North Carolina State University, 2002.
[9] B. George and L. Williams, "An Initial Investigation
of Test-Driven Development in Industry," in ACM
Symposium on Applied Computing, Melbourne, FL, 2003, pp.
1135-1139.
[10] A. Geras, M. Smith, and J. Miller, "A Prototype
Empirical Evaluation of Test Driven Development," in
International Symposium on Software Metrics (METRICS),
Chicago, IL, 2004, pp. 405 - 416.
[11] C.-w. Ho, M. J. Johnson, L. Williams, and E. M.
Maximilien, "On Agile Performance Requirements
Specification and Testing," in Agile 2006, Minneapolis, MN,
2006, pp. 47-52.
[12] M. Howard and S. Lipner, The Security Development
Lifecycle. Redmond, WA: Microsoft Press, 2006.
[13] IEEE, "IEEE Standard 610.12-1990, IEEE Standard
Glossary of Software Engineering Terminology," 1990.
[14] C. Larman and V. Basili, "A History of Iterative and
Incremental Development," IEEE Computer, vol. 36, no. 6,
pp. 47-56, 2003.
[15] R. C. Martin and R. S. Koss, "Engineer Notebook:
An Extreme Programming Episode,"
http://www.objectmentor.com/resources/articles/xpepisode.ht
m, no. 2001.
[16] E. M. Maximilien and L. Williams, "Assessing
Test-driven Development at IBM," in International
Conference of Software Engineering, Portland, OR, 2003, pp.
564-569.
[17] N. Nagappan, E. M. Maximilien, T. Bhat, and L.
Williams, "Realizing Quality Improvement Through Test
Driven Development: Results and Experiences of Four
Industrial Teams," Empirical Software Engineering, vol. 13,
no. 3, pp. 289-302, June 2008.
[18] P. Runeson, "A Survey of Unit Testing Practices,"
IEEE Software, no. pp. 22-29, July/Aug 2006.
[19] J. Sanchez, L. Williams, and M. Maximilien, "A
Longitudinal Study of the Test-driven Development Practice
in Industry," in Agile 2007, Washington, DC, pp. 5-14.
[20] M. Siniaalto and P. Abrahamsson, "A Comparative
Case Study on the Impact of Test-Driven Development on
Program Design and Test Coverage," in International
Symposium on Empirical Software Engineering and
Measurement, Rio de Janiero, Brazil, pp. pp. 275-284.
[21] B. Smith and L. Williams, "A Survey on Code
Coverage as a Stopping Criterion for Unit Testing," North
Carolina State University Technical Report TR-2008-22,
2008.
[22] L. Williams and R. Kessler, Pair Programming
Illuminated. Reading, Massachusetts: Addison Wesley, 2003.
[23] L. Williams, E. M. Maximilien, and M. Vouk,
"Test-Driven Development as a Defect-Reduction Practice,"
in IEEE International Symposium on Software Reliability
Engineering, Denver, CO, 2003, pp. 34-45.
Appendix
Here we provide the questions asked of the
developers and testers in our surveys.
A.1 Developer Survey
1. When you develop code, do you work from a
design document?
o Yes
o No
2. If you use a design document, is it a high level
design document?
o Yes
o No
3. Approximately how many lines of code (LOC) do
you write before you start writing DRTx/unit tests for
it? [open ended]
o The structure of the code
4. Approximately what percentage of your unit tests
are automated? [open ended]
5. What is the typical source lines of code to test
lines of code ration for your code? For example, 0.3
would mean that for 100 lines of source code you have
30 lines of test code. [open ended]
6. Which of these do you think about in your unit
tests? (check all that apply)
o Security
o Reliability
o Functionality
o Performance
o Other
7. What stopping criteria do you use to stop writing
unit tests? [open ended]