Previous Next

Critical Perspectives on the Scholarship of Assessment and Learning in Law

Preface

Craig Collins, Vivien Holmes and Paul Maharg

In this Preface to the first volume of the series Assessment in Legal Education, we outline the scope of the series, the reasons for its development and the ways it may assist those involved with legal education generally.

Assessment is a considerable and expanding disciplinary sub-domain in legal education. The processes by which law schools make judgments upon their students is one of the most important activities that law school staff undertake, with effects that can be long-lasting on their students. And yet there are few aspects of legal education that are more controversial and confronting as assessment, or as varied in practice, theory and results. In law schools throughout the Common Law world there are conservative practices derived from models of literacy and knowledge resumption that can be traced back to 19th-century models of assessment.1 There are also many emergent new practices, which arise from rejuvenated older practices in legal education itself, or from multidisciplinary borrowings and transplants, a few of which involve innovative uses of digital technologies.2 And beyond Law there is a substantial and fast-expanding literature on assessment in school education, in adult learning, university education and in professional learning; and legal educators in recent decades have increasingly drawn upon this diverse literature in legal education and its practices.3

In all of this, transfer of knowledge about assessment can be problematic, from one disciplinary domain to another, from the legal academy to the legal profession, and from one jurisdiction to another. In addition, many assessment practices stem from jurisdictional customs and are often strongly associated with a jurisdiction’s established views of learning, teaching and curriculum design. We may come to see assessment differently if we move beyond a Westphalian view of our apparently separate jurisdictions and appreciate what is happening in other jurisdictions, where assessment is almost certainly designed for different learning situations, from different cultural assumptions, and in the midst of different economic factors. Our perception of assessment theory and practice can grow when we encounter new forms of assessment, or fresh theoretical advances, or when we see familiar forms of assessment such as essays or reports deployed in unfamiliar contexts, or put to new and interesting purposes.

This series therefore offers views of assessment in legal education across a range of Common Law jurisdictions. Each volume will provide:

  • Information on assessment practices and cultures within a jurisdiction.
  • A sample of innovative assessment practices and designs in a jurisdiction.
  • Insights into how assessment can be used effectively across different areas of law, different stages of legal education and, where relevant, the implications for regulation of legal education assessment.
  • Appreciation of the multi-disciplinary and interdisciplinary research bases that are emerging in the field of legal education assessment generally.
  • Analyses and suggestions of how assessment innovations may be transferred from one jurisdiction to another.

‘Assessment’ in this series therefore applies to the assessment of learners – not just the assessment of students, but all who undertake legal study and whose performance is evaluated. It does not apply to the evaluation of teachers or of law schools, for, critical as these topics are to legal education, they involve significantly different literatures, contexts and approaches.

Our series methodology has been designed to be as open as possible in order to accommodate as many cross-cultural, ethnographic, educational and legal issues as possible. The series does not attempt comprehensive listing of assessment practices in a jurisdiction or across jurisdictions. While statistical evidence is much needed in legal education (where, by comparison with disciplines such as medical education, there are very few and reliable datasets), a quantitative global research project is a major undertaking, both in the initial data collection and the updating of the data collected if the dataset is to be useful as a reliable, continuous and contemporary resource. Our concept of scholarship is also open, involving diverse bodies of theory from many disciplines, including the ground-up theory that emerges from assessment practices in law schools and elsewhere. As our subtitle suggests, therefore, we take a critical perspective not only on assessment theory but on assessment practice too.

The series has also taken a different approach to that of other international legal education initiatives, such as the Internationalisation of Legal Education. In the book of that name, the editors Christophe Jamin and William van Caenegem provided a snapshot of the debates surrounding this subject by issuing a questionnaire to a wide range of jurisdictional reporters, 38 in total, who each authored a National Report. The collected reports were then collated into the book’s substantial General Report, authored by the editors, and presented to the Vienna Congress of the International Academy of Comparative Law, in July 2014. Some 19 of the 38 National Reporters wrote up their reports for the volume that was published by Springer. The General Report was a substantial contribution to our knowledge of the internationalisation of legal education.

Our focus in this project is different, however. We wish to give attention to the design of innovative assessment in legal education – a topic at once much more particular than internationalisation, in its focus on assessment, and also broader, in its focus on innovation. As series editors, we do not attempt to define innovation, for that would be to define a concept from our own standpoint as cultural subjectivities beyond the jurisdiction and scope of each volume. Instead, we leave the decision to those editors of the jurisdictional volume who will have specific knowledge of the assessment practices, the bodies of theory and the more general legal education practices in the jurisdiction. Nor are the series editors attempting closely comparative accounts of legal education assessment. As a methodology, comparativism is essential to law and legal education in a global world. Comparativism itself is undergoing change, moving from a methodology grounded in private law conceptions of legal families to constitutional issues, human rights and judicial review. Its empirical methodology is changing too, moving away from functionalist concerns, and becoming more heterogeneous and interdisciplinary in its methods.4

We therefore draw upon the more open structure of comparativist studies.5 Our focus on innovation, diversity and variety of assessment practices means that we want our collection of data to be as open-ended, as diverse and as variegated as is required. Who would determine that? We would answer that those working in an editorial role inside legal education within a jurisdiction are best placed to understand issues, theory and practices from within, and with a sense, too, of what is beyond the jurisdiction. But if this international series does not set out to be a comparative jurisdictional project, it does attempt to embed important insights of comparativist theory and practice in its work. It accepts Frankenberg’s bracing critique of the conventional comparativist approaches, sides with Siems on the value of fresh critical approaches, and attempts to discover and critically discuss innovation in a jurisdiction, where and when it happens, to what effects and in which contexts.6

As series editors, we therefore encourage the jurisdictional editors to challenge their own and their authors’ assumptions, their ‘hidden curricula’ (to adopt a term of art from education), their unstated educational and assessment norms. We encourage ethnographical, anthropological approaches, as well as more generally accepted educational analyses of assessment. We draw upon the diversity of methods in education itself – indeed we make a strong argument for interdisciplinarity in our treatment of assessment, without underestimating the difficulties of such an approach. Other disciplines may show how this can be achieved. In his groundbreaking study of the material culture of experimental microphysics, the historian of science Peter Galison investigated how the many professional groups involved in that domain (computer designers and programmers, engineers, physicists, instrument makers, policy-makers, politicians, university management) could communicate to share knowledge collaboratively on projects. According to Galison, they ‘traded’ concepts and language, and they coordinated across disciplines without homogenising, such that as trading partners in research projects they could ‘hammer out a local co-ordination despite vast global differences’.7

Galison makes a strong argument for the presence of at least some understandings across disciplines in the success that attends such endeavours, without which there could be no communication let alone successful completion of projects. We hold that the same can be true comparatively, within legal education, across disciplines and across jurisdictions. Dialogue is possible in the ‘trading zone’, to adopt Galison’s vivid metaphor, and often enables deeper dialogue in further border crossings.

We seek to encourage dialogue therefore; but not the dialogue that will merely reproduce forms of assessment. Instead we seek to explore innovation in assessment processes, methods and results that may bring about transformation in assessment in legal education, for students, staff, law schools, regulators and others. We also recognise that the attempt to transplant, to reproduce forms of learning and assessment, often contains hidden values more akin to ‘imperialism and a colonialism under the guise of supposedly value-free or objectively universal terms’.8 It is questionable whether such reproduction can be carried out without some degree of change and development. In this respect, reproduction often contains the seeds of its own transformation, as Henry Giroux observed: ‘reproduction is a complex phenomenon that not only serves the interest of domination but also contains the seeds of conflict and transformation’.9

Finally, we would make the observation that regulation and assessment are often intimately bound up with each other, and in terms of assessment we need to turn our gaze to the effects that regulation has on assessment. Regulators increasingly interpret their role as the safeguarders of public interest, concerned with risk, and balancing the forces of conservation and innovation. Assessment figures largely in their thinking and regulatory practices. And yet detailed resumption and analysis of the educational evidence, it is probably fair to say, is lacking in most legal educational reports in many jurisdictions. In England, the Legal Education and Training Review (LETR) Report pointed to the significant absence in the primary legal educational literature of substantial research upon which new educational practices could be founded, or upon which bases older practices could be confirmed as effective and further developed, or confirmed as ineffective in specific contexts, and laid aside.10

The result is that regulators may have little sense of which assessment practices are effective, why and in which contexts. Generalisations can thus be upheld; received opinion remains uncontested; normal categories of educational practices can remain unchallenged. Much more research needs to be carried out, in most jurisdictions, and confirmatory studies also need to be developed. We hope that this series will contribute to that literature.

This series is published via the PEARL Centre (Profession, Education and Regulation in Law), in The Australian National University’s College of Law, School of Legal Practice. The centre produces research upon the legal profession, on legal education and on the regulation of both. The centre decided to produce a book series for a number of reasons. From discussions with academics and with professional bodies in Scotland, Ireland, England, America and Australia, it was clear to Maharg and others that, while there were innovations in assessment in every jurisdiction, information and description tended to remain in the jurisdiction, and was seldom visible beyond it. Innovation is complex and difficult enough in itself; but once designed and implemented, its dissemination can be even more difficult. Once the platform of the PEARL Centre was formed at The Australian National University College of Law in 2015, its interdisciplinary and inter-jurisdictional focus made it the natural home for a book series that sought to be global and local at the same time, and to support experimentation, innovation, critical discussion, theory construction and effective practices in educational assessment.

Each volume in the series will be edited by at least one editor from the jurisdiction under consideration, and who will work with the series editors in the production of the volume. Editorial decisions regarding the choice of chapter subjects will be left largely to volume editors; and where possible the collection of chapters will be preceded by a call for papers or a workshop or conference at which chapters can be presented as drafts for discussion. The Introduction to each volume will provide a substantial overview of the salient issues affecting assessment theories, practices and cultures in the jurisdiction, while the volume editors will be encouraged to commission at least one chapter that focuses on issues of legal education regulation in the jurisdiction.

It is intended that the following volumes will be produced:

Vol

Jurisdictions

Approx. date of production

1

England

2019

2

Hong Kong, Singapore, Ireland

2020–21

3

Canada

2020–21

4

Australasia

2022

5

USA

2023

One volume will comprise three smaller jurisdictions, namely Hong Kong, Singapore and Ireland, where the varied political, economic and cultural contexts of small jurisdictions will provide a rich source of comparison. The rest comprise a variety of single jurisdictions (England) or multiple jurisdictions of states, territories, and provinces (Australia, Canada, USA). We are of course acutely aware that there are few Asian, and no African or South American, Common Law jurisdictions included. Our series was limited in terms of resource and is, after all, an experiment; and should we have more resource in the future then further Common Law jurisdictions could be the focus of a second series.

This series is designed to give a sense of what assessment practices appear to be across a range of jurisdictions. We hope that they will be useful for those seeking a summary of the contemporary issues facing academics, students, regulators, lawyers and others in the jurisdictions under analysis. We hope, too, that the exemplar chapters may assist cross-jurisdictional fertilisation of ideas and practices.

Finally we hope that the series as a whole, with its rhythms of overarching introductions and its exemplar chapters, may be a useful model for other areas of legal education. This may be a fond hope; but in its small ambition the series at least makes a gesture to the future. Hand in hand with a comprehensive historical analysis of assessment in these jurisdictions – no small project – it might contribute to dialogue between all those affected by assessment in legal education, and the mapping of the research domain.

References

Abrams JR, ‘Experiential Learning and Assessment in the Era of Donald Trump Drafting Statutes and Rules: Pedagogy, Practice, and Politics: Symposium Articles’ (2017) 55 Duquesne Law Review 75.

Arthurs H, ‘The World Turned Upside down: Are Changes in Political Economy and Legal Practice Transforming Legal Education and Scholarship, or Vice Versa?’ (2001) 8 International Journal of the Legal Profession 11. doi.org/10.1080/09695950120103154.

Barton K, McKellar P and Maharg P, ‘Authentic Fictions: Simulation, Professionalism and Legal Learning’ (2007) 14 Clinical Law Review 143.

Bone A and Hinett K, Assessment for Learning: A Guide for Law Teachers (UK Centre for Legal Education 2002).

Cunningham L, ‘Building a Culture of Assessment in Law Schools’ (Social Science Research Network 2018) SSRN Scholarly Paper ID 3216804 <https://papers.ssrn.com/abstract=3216804> accessed 21 October 2018.

Faulconbridge JR and Muzio D, ‘Legal Education, Globalization, and Cultures of Professional Practice Symposium: Empirical Research on the Legal Profession: Insights from Theory and Practice’ (2009) 22 Georgetown Journal of Legal Ethics 1335. doi.org/10.2139/ssrn.1516314.

Fenwick M, The Shifting Meaning of Legal Certainty in Comparative and Transnational Law (Mathias M Siems and Stefan Wrbka eds, 1st edition, Hart Publishing 2017).

Frankenberg G, ‘Stranger than Paradise: Identity & Politics in Comparative Law’ (1997) Utah Law Review 259.

Galison P, Image and Logic: Material Culture of Microphysics (University of Chicago Press 1997).

Gijbels D and others, ‘The Relationship between Students’ Approaches to Learning and the Assessment of Learning Outcomes’ (2005) 20 European Journal of Psychology of Education 327.

Giroux H, Ideology, Culture and the Process of Schooling (Temple University Press 1981).

Hamm MS, Madison BV and Murnane RP, ‘The Rubric Meets the Road in Law Schools: Program Assessment of Student Learning Outcomes as a Fundamental Way for Law Schools to Improve and Fulfill Their Respective Missions’ (Social Science Research Network 2018) SSRN Scholarly Paper ID 3158461. doi.org/10.2139/ssrn.3158461.

James NJ, ‘Power, Knowledge and Critique in Australian Legal Education 1987–2003’ (DPhil thesis, Queensland University of Technology 2004) <https://eprints.qut.edu.au/15910/1/Nickolas_James_Thesis.pdf> accessed 21 September 2018.

Jones R, ‘Assessment and Legal Education: What Is Assessment, and What the Does It Have to Do with the Challenges Facing Legal Education Symposium: The State and Future of Legal Education’ (2013) 45 McGeorge Law Review 85.

Katz DM, ‘The MIT School of Law – A Perspective on Legal Education in the 21st Century’ (2014) 2014 University of Illinois Law Review 1431.

Keyes M and Johnstone R, ‘Changing Legal Education: Rhetoric, Realty, and Prospects for the Future Special Issue: Teaching and Scholarship’ (2004) 26 Sydney Law Review 537.

Kift SM, ‘Harnessing Assessment and Feedback to Assure Quality Outcomes for Graduate Capability Development: A Legal Education Case Study’ in Peter L Jeffery (ed), Australian Association for Research in Education (AARE) 2002, December 2002, Brisbane, Queensland, Australia <https://eprints.qut.edu.au/7474/> accessed 21 October 2018.

Maharg P, ‘The Culture of Mnemosyne: Open‐book Assessment and the Theory and Practice of Legal Education’ (1999) 6 International Journal of the Legal Profession 219. doi.org/10.1080/09695958.1999.9960464.

——, Transforming Legal Education: Learning and Teaching the Law in the Early Twenty-First Century (Routledge 2007).

Peters A and Schwenke H, ‘Comparative Law Beyond Post-Modernism’ (2000) 49 International & Comparative Law Quarterly 800. doi.org/10.1017/s0020589300064666.

Ribstein LE, ‘Practicing Theory: Legal Education for the Twenty-First Century Symposium: The Future of Legal Education’ (2010) 96 Iowa Law Review 1649.

Sheldon KM and Krieger LS, ‘Understanding the Negative Effects of Legal Education on Law Students: A Longitudinal Test of Self-Determination Theory’ (2007) 33 Personality and Social Psychology Bulletin 883. doi.org/10.1177/0146167207301014.

Siems M, Comparative Law (2nd edition, Cambridge University Press 2018).

Sugarman D, ‘Beyond Ignorance and Complacency: Robert Stevens’ Journey through Lawyers and the Courts’ (2009) 16 International Journal of the Legal Profession 7. doi.org/10.1080/09695950903354840.

Van Hoecke M and Warrington M, ‘Legal Cultures, Legal Paradigms and Legal Doctrine: Towards a New Model for Comparative Law’ (1998) 47 International & Comparative Law Quarterly 495. doi.org/10.1017/S0020589300062163.

Webb J and others, ‘Setting Standards. The Future of Legal Services Education and Training Regulation in England and Wales’ (SRA, BSB, IPS 2013).


1 There is no single history of legal education assessment across jurisdictions, indeed no histories of assessment in a single Common Law jurisdiction. The conventional nature of much legal education assessment, however, is noted in many studies, often as a standard refrain describing the status quo before offering descriptions of innovation. Such a rhetorical strategy should be viewed with suspicion of course; but it is remarkable how certain forms of learning and assessment appear repeatedly in law school curricula over the long 20th century. With isolated and notable exceptions it is only in the last 30 years or so that there has arisen a literature critical of assessment methods and the lack of both assessment innovation and theory. Across a range of jurisdictions and topics the following is a sample of the literature: Kennon M Sheldon and Lawrence S Krieger, ‘Understanding the Negative Effects of Legal Education on Law Students: A Longitudinal Test of Self-Determination Theory’ (2007) 33 Personality and Social Psychology Bulletin 883, doi.org/10.1177/0146167207301014; Ruth Jones, ‘Assessment and Legal Education: What Is Assessment, and What the Does It Have to Do with the Challenges Facing Legal Education Symposium: The State and Future of Legal Education’ (2013) 45 McGeorge Law Review 85; Mary Keyes and Richard Johnstone, ‘Changing Legal Education: Rhetoric, Reality, and Prospects for the Future Special Issue: Teaching and Scholarship’ (2004) 26 Sydney Law Review 537; David Gijbels and others, ‘The Relationship between Students’ Approaches to Learning and the Assessment of Learning Outcomes’ (2005) 20 European Journal of Psychology of Education 327; Jamie R Abrams, ‘Experiential Learning and Assessment in the Era of Donald Trump Drafting Statutes and Rules: Pedagogy, Practice, and Politics: Symposium Articles’ (2017) 55 Duquesne Law Review 75; Larry Cunningham, ‘Building a Culture of Assessment in Law Schools’ (Social Science Research Network 2018) SSRN Scholarly Paper ID 3216804 <https://papers.ssrn.com/abstract=3216804> accessed 21 October 2018; Marie Summerlin Hamm, Benjamin V Madison and Ryan P Murnane, ‘The Rubric Meets the Road in Law Schools: Program Assessment of Student Learning Outcomes as a Fundamental Way for Law Schools to Improve and Fulfill Their Respective Missions’ (Social Science Research Network 2018) SSRN Scholarly Paper ID 3158461 <https://papers.ssrn.com/abstract=3158461> accessed 21 October 2018; Alison Bone and Karen Hinett, Assessment for Learning: A Guide for Law Teachers (UK Centre for Legal Education 2002); Sally M Kift, ‘Harnessing Assessment and Feedback to Assure Quality Outcomes for Graduate Capability Development: A Legal Education Case Study’ in Peter L Jeffery (ed), Australian Association for Research in Education (AARE) 2002 (December 2002, Brisbane, Queensland, Australia) <https://eprints.qut.edu.au/7474/> accessed 21 October 2018; Paul Maharg, ‘The Culture of Mnemosyne: Open‐book Assessment and the Theory and Practice of Legal Education’ (1999) 6 International Journal of the Legal Profession 219. doi.org/10.1080/09695958.1999.9960464.

2 The literature on such new theory and practices is extensive and growing, and it is one of the aims of this series to provide a guide to that literature as well as illustrations of innovative practices from a range of jurisdictions. See, for example, David Sugarman, ‘Beyond Ignorance and Complacency: Robert Stevens’ Journey through Lawyers and the Courts’ (2009) 16 International Journal of the Legal Profession 7, doi.org/10.1080/09695950903354840; James R Faulconbridge and Daniel Muzio, ‘Legal Education, Globalization, and Cultures of Professional Practice Symposium: Empirical Research on the Legal Profession: Insights from Theory and Practice’ (2009) 22 Georgetown Journal of Legal Ethics 1335. doi.org/10.2139/ssrn.1516314; Nickolas John James, ‘Power, Knowledge and Critique in Australian Legal Education 1987–2003’ (DPhil thesis, Queensland University of Technology 2004) <https://eprints.qut.edu.au/15910/1/Nickolas_James_Thesis.pdf> accessed 21 September 2018; Harry Arthurs, ‘The World Turned Upside down: Are Changes in Political Economy and Legal Practice Transforming Legal Education and Scholarship, or Vice Versa?’ (2001) 8 International Journal of the Legal Profession 11, doi.org/10.1080/09695950120103154; Larry E Ribstein, ‘Practicing Theory: Legal Education for the Twenty-First Century Symposium: The Future of Legal Education’ (2010) 96 Iowa Law Review 1649; Karen Barton, Patricia McKellar and Paul Maharg, ‘Authentic Fictions: Simulation, Professionalism and Legal Learning’ (2007) 14 Clinical Law Review 143; Daniel Martin Katz, ‘The MIT School of Law – A Perspective on Legal Education in the 21st Century’ (2014) 2014 University of Illinois Law Review 1431.

3 The research literature on this is too extensive to quote selectively here. The sheer number of journals and articles, and the increasing specialisation and focus upon many new forms of assessment and digital technologies is evidence of this. One might consider as an example the sophistication of organisations in medical education such as the Association of Medical Educators in Europe (AMEE) (discussed in the Introduction below), whose website contains a considerable array of information on assessment in medical education – brief guides, policy documents, research articles, systematic surveys of research.

4 See for instance Mark Van Hoecke and Mark Warrington, ‘Legal Cultures, Legal Paradigms and Legal Doctrine: Towards a New Model for Comparative Law’ (1998) 47 International & Comparative Law Quarterly 495. doi.org/10.1017/S0020589300062163; Anne Peters and Heiner Schwenke, ‘Comparative Law Beyond Post-Modernism’ (2000) 49 International & Comparative Law Quarterly 800. doi.org/10.1017/s0020589300064666.

5 See, for example, Matthias Siems, Comparative Law (2nd edn, Cambridge University Press 2018).

6 See Gunter Frankenberg, ‘Stranger than Paradise: Identity & Politics in Comparative Law’ (1997) Utah Law Review 259; Mark Fenwick, The Shifting Meaning of Legal Certainty in Comparative and Transnational Law (Mathias M Siems and Stefan Wrbka eds, 1st edition, Hart Publishing 2017).

7 Peter Galison, Image and Logic: Material Culture of Microphysics (University of Chicago Press 1997) 783, quoted in Paul Maharg, Transforming Legal Education: Learning and Teaching the Law in the Early Twenty-First Century (Routledge 2007).

8 Frankenberg (n 6) 269.

9 Henry Giroux, Ideology, Culture and the Process of Schooling (Temple University Press 1981) 109.

10 Julian Webb and others, ‘Setting Standards. The Future of Legal Services Education and Training Regulation in England and Wales’ (SRA, BSB, IPS 2013) xii, paras 1.30, 7.77.


Previous Next