Assessing Assessment: the Multiple-Choice Exam, the Impromptu Essay, and the Portfolio

This week’s readings for Theories of Assessment trace the very recent (mostly because of its short history) interest in speculating how a reviewer should go about assessing student writing. The “reviewer” seems to also be a subject of controversy in these articles, as scholars debate whether or not educators, students, or test raters are best qualified to assess student writing. Nonetheless, it is clear that theories of assessment have changed dramatically in the span of twenty years this sample of articles portray and stability for assessment seems nowhere near being close at hand.

Huot begins with an introduction to writing assessment and its academic research as it was known and studied from about 1975-1990. In general, Huot’s work outlines the rise of direct writing assessment research during this time period and also briefly attempts to situate it alongside its counterpart—indirect writing assessment—which flourished until the mid-1960s. Describing indirect writing assessment as the evaluation of student writing ability based on “examinations on grammar and usage” devoid of critical review by “independent readers” (237), Huot goes on to list the three main procedures for direct writing assessment: primary trait, analytic, and holistic. Not surprisingly, the holistic procedure is named the most popular because of its cost-effectiveness (such as I suspect it still is today) even though Huot warns it is not always the most appropriate.

The bulk of Huot’s article then reviews three main interests for direct writing assessment research and argues for or against each interest’s findings, speculating where the future of writing assessment will progress into the future. Essentially, these three interests include topic development and task selection, text and writing quality, and influences on rater judgment of writing quality. For topic development and task selection, Huot concludes “structure, wording, and overall presentation of a writing assignment can sometimes have important consequences within particular writing contexts” (246) and also that very little has been done in exploring these influences. For text and writing quality, he suggests that recent (to 1990) findings in linguistics and discourse analysis has seen a shift from interest in syntax to interest in “global-level language features” (250), speculating that the future of writing quality assessment lies in the hands of future discourse-level research and interests. Finally, for influences on rater judgment of writing quality, Huot makes very interesting conclusions on the inconsistency of research and need for further study in the area. Though he does remark that a majority of the literature address “content and organization” (256) as important factors on rater judgment, Huot predicts that a rise in the popularity of portfolio writing will begin to change the way scholarship studies the influences on rater judgment of writing quality.

Following almost exactly where Huot leaves off, White takes us to the relative present-day understanding of assessment as it is understood through portfolio scoring. Drawing attention to faulty current interest in holistic scoring of student portfolios, White suggests a new (“Phase 2”) method of scoring portfolios that sounds to me quite similar to Huot’s identification of primary trait assessment. This method adds to the portfolio requirement a clear statement of goals for each sample of writing on the part of the faculty and a cover letter submitted on top of the portfolio that rhetorically argues why the student believes the portfolio attached meets or does not meet the aforementioned statement of goals. White believes this creates a beneficial, more practical, and cost-effective way for raters to not reevaluate work that has already been graded by faculty members but instead critique the student’s effectiveness in arguing and supporting through the cover letter that the portfolio completes the goals set of them.

Following Huot and White’s rather dense and informative articles, Royer and Gilles take us back to the 1990s with an enjoyable anecdote revealing another radical idea stemming from assessment theory: allowing students to choose their own course of action in placement for freshman writing classes. Called “directed self-placement,” Royer and Gilles suggest this method pleases every party involved (student, teacher, administrator, etc.) and puts the burden of assessing a student’s ability’s squarely in the hands of the student him/herself. Personally, I found the idea to be incredible. I think it could be potentially dangerous, though, to have no requirements distinguishing basic writing from typical freshman writing courses apart from a student’s choice. If a director could rhetorically convince students who need basic writing to enroll in those classes I very much see the benefits (a more relaxed sense of belonging in basic writing, a student’s feeling of personal endowment in the classroom, and the such); however, unless those students who we were certain needed basic instruction take it we may be toying with unstable young adult feelings.

The report of the NCTE on SAT and ACT writing tests provides an eye-opening (but really not surprising) account of the tests ineffectiveness in assessing student writing. Harkening back to Huot’s “early days of assessment,” the NCTE Task Force not only concludes that the SAT encourages superficial, formulaic, non-critical writing on its tests but also risks encouraging the same vapid writing in the writing classroom. The Task Force also found that these tests continue to influence conventional “correctness” as “good” writing, favor certain ethno-diverse groups over others, and influence placement in university programs which are in direct conflict with the test’s purpose of measuring student ability for college admission, not college performance.

White ends the week’s readings by rebutting NCTE’s attack on single-sitting essay writing and attempts to expose it as a means to support the “currently” popular portfolio assessment method taking up arms at universities that can afford it. Notice that White’s “rebuttal” comes in 1995 while the NCTE report on the “new and improved” SAT/ACT structure comes in 2005. It is clear the NCTE has stood firm on this issue for over a decade, and it is unfortunate that nobody seems to be listening. White argues again and again that the multiple-choice and single-sitting essay’s prevalence, cost-effectiveness, and ease of use in college assessment seem to point to the fact that they are the tools to use; however, I think it is in kindergarten when we learn that just because something is popular it doesn’t automatically make it right. White’s logic is dangerously faulty in this piece and seems to almost settle for essay writing since it’s better than multiple-choice tests and cheaper than portfolio assessment. I don’t agree with this argument at all, and I certainly hope that he is wrong. Portfolio writing may not be the best method of assessment, but it a step forward, not one or two steps back.

Reflection on Website Project & Final Website URL

URL: ceiwertz.pcriot.com (dead link)

Reflecting on the complete experience of creating a professional website, I have to admit it was a lot harder to do than I had initially thought. I explored many options on the tools available to both create and host the website. I started off exploring Dreamweaver, but since I had a program at home called iWeb on my mac computer, I decided to use iWeb for designing the site since I could use it without driving to school or downloading a trial version of the software.

I also explored a lot of different free web servers for hosting the finished product of my website, and after going through quite a few I came across x10hosting.com, which offers free web hosting and a lot of freedom in uploading and maintaining the website. I think they even offer free email accounts attached to your website if you want to sign up for them. Another feature I liked about x10hosting is that they can transfer seamlessly to another domain if you ever buy one, which I also looked into. Luckily, no one owns ceiwertz.com, so I could purchase it for a year for only $10, but I decided for now I’ll stick to the subdomain ceiwertz.pcriot.com, which is free, until the day I might be able to publish the website on … I don’t know … a university’s domain?

After securing a program for designing the site and the host server to publish the site, I think came the hardest part: creating the site from scratch. Outlining which pages to include, especially given that I am the one with least amount of experience teaching in our class, was difficult; however, I have to say the finished product is one I am really proud of and one I feel reflects my talents and goals well. I especially love how the site has become something I think I will continually use as a professional. All my work can be stored on the site as PDF, which is amazing since I usually have multiple areas where I’m storing papers and conference information and the such. And, my links page is a quick way for me to navigate the web how I choose, allowing me to remember to look into certain conferences or follow certain journals like the online edition of Computers and Composition, which cannot be obtained in hard copy.

As the project pertains to our readings this semester in class, I come back again and again to Sullivan’s “Taking Control of the Page: Electronic Writing and Word Publishing.” Sullivan argues that computers have changed what we must be and what we must be able to do as writers, specifically calling word processing as a writer an act of both writing and publishing. Like this article, publishing a website has made me realize that the act seems to not only change what I must do as a writer but also what I must do to be literate in today’s society. If I am to market myself as an academic professional—especially in composition rhetoric—I need to be able to compile my studies, CV, links, and the such in a website. If I do not have these tools in my tool belt, so to speak, I am not nearly as marketable as a professional who can.

I also think about how concerned we have been this semester about the composition classroom and writing as a skill versus an art. Beginning from scratch on this project, because I am by no means a talented website designer, reminds me that to reach the polished, finished project we must have both acquired a skill (which I did to some extend during the course of this project) and utilized our voice in artistically making a product our own creation. The same is true for writing. It is a skill and one that needs to acquire to some extent before making it their own. I suppose the question we have been seeking to answer this semester is what is more important, then, in a freshman writing classroom: acquiring a skill or developing the art. Perhaps that is a question that cannot be answered because every class will have different goals and different students, but I would hope that at the end of any semester we as writing teachers have taught that there is more to composition than technicalities, just as in the project of creating a website I have discovered there is much more to web design than web layout or web hosting.

WEBSITE!!

Here's the link to my website. The homepage is what needs the most work: I'm going to include a short bio where the jibberish is now and also going the have a professional picture where the picture of the lake is. Let me know what you all think!

http://ceiwertz.pcriot.com

The Fight for Freedom: Critical/Cultural Studies, the Writing Classroom, and Ethical Teaching in the University Context

The collection of readings for this week—grouped under the title Critical and Cultural Studies Pedagogy—expands the theories of CCS that Berlin touched on last week. Together, they examine the foundations, rise, and

Freire, writing in 1970, gives us our earliest sample for the week. Inspirational and steeped in thought-provoking theory, Freire’s piece gives shape to early foundations behind cultural studies by grounding it firmly in the notion of freedom. If an educator teaches methods for defining reality, then—Freire writes—“it is a courageous endeavor to demythologize reality, a process thought which men who had previously been submerged in reality begin to emerge in order to re-insert themselves into it with critical awareness” (621). Working with adult illiterates, Freire’s main goal is to examine and teach cultural literacy. His work outlines several tools and phases with which he implements this goal, but I believe his greatest accomplishment is situating cultural literacy (in this case that of Latin America) as work in both academic and equality.

Berlin, like Freire, also draws a link between instruction and enforced ideology. Writing now in 1988 (four years later than the article we read for class last week and eighteen years after Freire), Berlin classifies the teaching models of rhetoric into three different classes, each with a unique ideology on reality: cognitive, expressionistic, and social-epistemic

First, cognitive rhetoric defines “the real as the rational” (723). It divides writing into three stages: planning (further divided into generating, organizing, and goal setting); translating; and reviewing (both evaluating and revising). As such, cognitive rhetoric follows the theories of Flower and Hayes, Emig, and Bowel and Gintis. As Berlin suggests, it follows the epigram: “The existent, the good, and the possible are inscribed in the very nature of things as indisputable scientific facts, rather than being seen as humanly devised social constructions always remaining open to discussion” (725).

Second, expressionistic rhetoric argues that “the existent is located within the individual subject. While the reality of material, the social, and the linguistic are never denied, they are considered significant only insofar as they serve the needs of the individual” (726). Theorists within this class include Elbow, Lutz, and Murray.

Third, social-epistemic rhetoric, the most recent, is spearheaded by Burke, Bartholomae, Young, Bizzell, among many others. Within this camp, “the real is located in a relationship that involves the dialectical interaction of the observer, the discourse community … and the material conditions of existence” (730). Therefore, the real is never actually defined, but rather “posited” as “a product of the dialectic in which all three come together” (731-31). Chosen by Berlin as “the most worthy of emulation in the classroom” (735), social-epistemic rhetoric puts the question of ideology at the center of teaching writing and allows students to see the forces of society, culture, and self working within all writing.

Writing one year earlier than Berlin, Shor argues that Freirean notions of critical theory do not simply apply to the illiterate working classes of third-world countries but also have very important and extreme need to be implemented in thriving metropolises of America as well. Implementing dialogue as the key to unlocking critical literacy, Shor believes, instructs students in the quickest and most efficient manner. Through language, the “work” of the country becomes critical and critical thinking becomes much less remote.

Hairston, examining the current stage of writing as a discipline in 1992, portrays CCS with much different brush strokes than earlier scholars so far this week. Interestingly, she sees CCS at its core a method for instilling “dogma” and “ideology” (698) onto reluctant and resistant students. Ironically, the ideology which seeks to unmask hidden ideology instills and silences a resistant student’s ideology … I know that’s repetitive, but it seems to (comically) sum up the injustice to which Hairston points. Writing has become, in Hairston’s address, a product of its housing in English departments: critical theory. I was actually pretty shocked while reading Hairston at her extreme disregard for critical theory. Claims such as, “our society’s deep and tangled cultural conflicts can neither be explained nor resolved by simplistic ideological formulas” and “we have no business [as writing teachers] getting into areas where we may have passion and conviction but no scholarly base from which to operate” (705). Wow. Her whole distaste of critical/cultural studies makes complete sense; however, seems to completely turn the goals of the discipline upside down for scholars of rhetoric. She asks: What ever happened to trying to make students better writers? Why are we wasting our time teaching students in a writing class about sexism, structuralism, deconstruction? These, she argues, have no place in the writing classroom. She, it seems, leans a bit toward expressionism in agreeing that our classrooms are more diverse but need to be focused on the student first and then the student’s place within his or her community. I believe her argument is extremely relevant, especially to myself who at least as of late was a big proponent of CCS; however, it is still steeped in criticism and seems to just follow the same process of past expressionists who place emphasis on student writing, student experience—trying to detach the educator from focusing on the political and social constructs that line every piece of writing.

Smith, too, and as an excellent final note to the week, reconsiders CCS in the contemporary college. Writing in 1997, he is our latest example of writing in our sample and ends with very provocative questions on gatekeeping. Drawing attention to the fact that we as a discipline seem to steer clear away from the notion of gatekeeping, Smith brings to full-view the notion that scholars are always in contention within a university setting of gatekeeping: consider, as Smith does, graduation. Thus, Smith suggests it is best as scholar of writing to suit a course to the context of the university, where what is taught is of benefit to all majors and all students, not just those interested in composition. I believe what Smith would have us see is that teaching CCS in the writing classroom hinders students much more than it helps them. Just as Hairston critiques, we need to focus on teaching writing as discipline, not liberation.

Teaching Writing, Teaching Reality

The readings for this week—grouped under the title Theories of Pedagogy—ask questions about ways in which we teach composition. Which pedagogy is most effective? Are experimental methods useful? How can we classify these pedagogies? Do all theories need to apply to the classroom setting? Whatever the question asked of by each scholar this week, each study seeks to expand the field of composition studies by re-envisioning the composition classroom and the main elements (writer, reader, teacher, reality, and the such) that make it work.

We open with George Hillocks, who presents a massive meta-analysis of over 500 experimental studies of the composition classroom between 1963 and 1982. Utilizing the sophistication and scientifically tangible results of statistical analysis, Hillocks brings a great deal of credibility to experimental studies in the writing classroom. For example, Hillock classifies his 60 sample studies into different categories depending on such elements as the mode of instruction in the classroom and the focus of instruction in the classroom. Using these classifications, Hillocks proves that classrooms which create an environmental mode of teaching (which, among other things, promotes both peer and teacher feedback to work and direct instruction to clearly-outlined goals) perform better than those which implement a natural mode (where teacher is simple facilitator of positive feedback) or individualized mode (where instruction is done on an individual basis, mostly by tutors).

Furthermore, Hillock proves something that must have been rather shocking for his day and audience: when studying the focus of instruction, Hillock proves that classrooms that study grammar and mechanics perform nearly a third lower than classrooms that study no grammar and mechanics. Taking up these controversial topics in the early 80s, Hillock paves the way for more experimental research in theories of pedagogy in a time when many thought they were to a large extent a waste of time and energy.

Going even a few years before Hillock, we then turn to Berlin who, writing in 1982, outlines the major pedagogical theories of the time. Interestingly, Berlin supports a classification of pedagogical theories not interested in the emphasis of one element of the composing process over the other (e.g. writer, reader, reality) but rather by the way in which these elements are envisioned and how they are changed from one theory to the other. Not surprisingly, then, most of his classifications are based on philosophical interpretations of rhetoric: the dominant Current-Traditional Rhetoric and his reactionary Neo-Platonic, Neo-Aristotelian, and New Rhetorics.

These divisions are not so new to me, but as I became reacquainted with these schools of thought I was surprised to see how much my own philosophies on rhetoric have changed over the past five to seven years. I recall first being introduced with the New Rhetoric and being totally won over. But now I see major compositional problems with viewing a writer as creator of truth and audience as shaper of the writer’s universe. Instead, I keep being yanked back into reader-response and wonder as this theory does about the relation between writer and reader, or more importantly about communities. If the writer is actually at most times a reader who actualizes her own text as her audience must do after publication then this would not place the creation of truth inside the writer as New Rhetoric does but in the community, in the writer as a reader. But I digress; Berlin’s contribution to our list of readers this week is that studying different pedagogies does not merely mean we study different subject areas over others, it means we begin to be taught different theories of reality, different theories of truth.

Our final three readers bring theories of pedagogy into the current century with Breuch writing in 2002, Fulkerson in 2005, and Downs in 2007.

Breuch finds problems with the contemporary (negative) view of that post-process approach to teaching composition, that process theory is simply a call for the codification of writing steps which can be taught in the composition classroom. Specifically, she argues that post-process theory contributes to theories of pedagogy not as a concrete application to the classroom but rather as a redefinition of writing as "an activity rather than a body of knowledge” and that “communicative interactions with students [are] dialogic rather than monologic" (98-99). Her work suggests that studying theories of pedagogy goes beyond simply applying study to the classroom; her work suggests a reexamining of the definitions and players within that classroom itself.

Fulkerson and then Down and Wardle argue that writing has dramatically changed since the turn of the new century. Fulkerson supports that writing in college has become a topic of great argumentation among composition scholars since the boom of cultural studies hit the university. Down and Wardle argue that teaching writing in the new century has become an opportunity to teach writing as a legitimate discipline. They suggest, too, that writing in the new century has become a battle to support one method of thinking over another; however, this thinking envisions writing as a sect of scholarship with a rich and diverse history with much to add to study at the university aside from its function in preparing students for the writing they must complete while in college.

Proposal for: Composition 2.0: Erasing Authorship and Considering (for Real) Communal Writing

In the opening to their most recent specially-edited edition of Computers and Composition (Jan 2010), Michael Day, Randall McClure, and Mike Palmquist call for a new understanding of Web 2.0 and the Web 2.0 Movement. This vision, which they name Composition 2.0, realizes Web 2.0 as it has had drastic effects on the composition process, collaboration within composition, and participatory writing. Embracing hypertext composition for its advances in online collaboration, scholarship both in and beyond this edition have explored the influence collaboration has taken on the writer; however, very little if anything has been done to examine the influence Web 2.0 and collaborative computer-mediated composition has taken in blurring the lines between authorship and readership. Johndan Johnson-Eilola, in his groundbreaking work Nostalgic Angels, supports a deconstructionist view of hypertext composition. Using the foundations build by his study, a social understanding of hypertext composition begins to realize the internet as an actualized deconstruction of society in which borders are easily broken and the rules which govern its composition are continually being renewed and altered by even the most unassuming participant. From these foundations a stronger understanding of hypertext authorship can be understood as the product of collaborative communities rather than the product of an individual who can be constantly and limitlessly censored.

This paper will argue that a collaborative view of hypertext composition reveals new roles for traditionally defined players such as writer and reader, roles which easily become obfuscated and create social space where authorship becomes the product of collaboration.

As such, this paper will suggest new ways of viewing online composition. As influenced by Johnson-Eilola, it will suggest a strongly social view of online collaborative efforts such as blogs and wikis. It will begin with a brief introduction on current pessimism in hypertext studies and then suggest a social view of computer-mediated composition. This social view, as well as seeing an important and unmistakable obfuscation of author and reader, will open up discussion for potential ways in which Web 2.0 (or Composition 2.0) has pulled apart the writing process and allowed for writing in ways never before imagined.

Furthermore, it will be of benefit to those interested in the field of computers and composition, especially to scholars who are working to understand the impact of social networking sites and online collaborative efforts on composition. It will also be of interest to individuals who are interested in the impact continued social studies of the internet have on composition in general and those who either agree or disagree with the statement that the internet has changed permanently altered the way writing is done in communities.

Literacy and Community: The Ties that Bind in Literacy Theory

This week’s readings—or rather the readings for our last class that I switched up with the readings for this week—examine the influence of literacy on daily life, community, and profession. It seems clear from the readings that literacy studies is a crucial subfield of composition studies that asks questions just as much of scholarship as it does of society. Looking at the readings as a whole, it appears as though literacy studies calls for a reexamination of literacy, teaching on a community-by-community basis. Heath, Hull, and Cushman suggest that looking within the community’s complex literacies for the best solution to literacy-inspired problems will create better employees, better citizens, and a better understanding of the field of literacy studies as a whole. Brandt and McCarthy argue that locally based sponsors/teachers dramatically enhance/detract from the student’s learning of literacy and literacy abilities. On a community level, these teachers shape the literacy of their students and with it their abilities in composition as well.

As our earliest sampled theorist for the week, Shirley Brice Heath explores the “protean” or ever changing, complex task of assigning one stage of literacy to any one community. Using data collected from the community of Trackton, located in the Carolinas, between 1969-79, Heath explores the influence of literacy as it comes into contact with life from all angles, and concludes that literacy events and the literacies of communities exist in many forms that cannot easily be categorized on the scale developed by Goody for cataloguing a community somewhere between oral and literate. Rather, communities like Trackton—and arguably all communities—encounter different stages of literacy through different literacy events in different social transactions over time. As Heath states, "the nature of oral and written language and the interplay between them is ever shifting, and these changes both respond to and create shifts in the individual and social meaning of literacy" (466). So Heath’s contribution to literacy studies is not purely a call to view literacy events as varied event to event in each community, she also offers a call to understand that literacy events occur within a social context, one which shapes and creates meaning for both the individual and the community itself. Lucille McCarthy, who writes five years after Heath, suggests such a study could also apply to students whose many different communities are each class they take. Each must be examined and its literacy understood and learned separate from the other. Since each course may have different goals and objectives, it is difficult to gauge its placement on a literacy scale, just as it is difficult to place a community as described by Heath on a literacy scale.

I believe this is a very humble study that could easily be applied to prejudice for/against the literate/illiterate as defined by the scale Heath mentions as described by Goody. It is very important to not simply label a community by one or two literacy events but far more useful to examine, as Heath does, the interplay of literacy and orality within each community and perhaps the extent to which traditionally high-level literate communities exhibit oral activity and high-oral communities embrace literacy events. This is the very topic Ellen Cushman brings this very idea to fruition in her article where she defends the importance of the oft-neglected lower classes in policy making. Particularly looking to faculty members as potentially large contributors to policy change, Cushman argues that literacy studies gives weight to arguments made outside of the upper-class ranks of government.

Deborah Brandt, like Heath, sets out to view a historical understanding of literacy with fresh eyes. Outlining the rise and fall of apprenticeships at small, decentralized print shops of the American antebellum, Brandt concludes that like the sponsors of these programs, contemporary writing scholars "haul a lot of freight for the opportunity to teach writing" (183). Scholars, she suggests, must carry the burden of ideological influence of their teaching, especially that suggested from the classic argument from business employers that students who graduate college do not know how to write. Brandt hopes, in her final statement, that her work promotes an understand of composition not as a field which prepares its students for the job markets they wish to enter but rather as one which assists and studies individuals in their pursuit of literacy, and also of the literacy that pursues them.

Glynda Hull turns her study on literacy to the workplace. Investigating the current understanding of literacy in the workplace, Hull looks beyond the engendered stereotypes to see the potential for new understandings of workforce potential and literacy in the workplace. Interestingly, Hull suggests the benefits of an apprenticeship-style program may aid in instructing employees in the literacy of a particular workforce that may be simple to some but inaccessible to others. This reminds me greatly of the work I have done while at Starbucks Coffee Company, which is steeply entrenched in a literacy of its own making. I have had many opportunities to train new employees and have often found that the most learning comes from watching a skill in progress as a silent observer, learning about the skill, observing once again, and then attempting to carry out the skill. Just as Hull suggests workers cannot assumed to be fully-literate in their workplace without seriously rethinking the ways they are trained, teaching a new employee at Starbucks the entire culture of coffee as well as the company’s spin on this culture is a weighty process that works best when tailored to fit the person learning the material instead of the company or person teaching the material.