Justice Byron R. White’s former law clerks remember him not as one of his generation’s greatest football players, but as one of its sharpest legal minds. He was, some say, the smartest person they ever met. Yet in 31 years on the Supreme Court, the most gifted scholar-athlete of his time made far less conspicuous a mark on the law than colleagues with far less potent intellects. One reason was the inelegant, cryptic, often-confusing writing style of the brusque, no-nonsense White. Another was a virtue now very much out of vogue: his modesty in the exercise of judicial power.

White, who died at age 84 on April 15, nine years after retiring from the Court, was the last justice who not only preached but also consistently practiced judicial restraint. In an era when fame favors jurists who aggrandize their own powers to further favorite causes, White was more disposed to defer to the elected representatives of the people. He rejected the urge to impose personal "value preferences" by inventing constitutional rights, and he disdained such urges even more than he disliked the nickname "Whizzer." He was no liberal. Nor was he a conservative, except in the old-fashioned sense of applying the Constitution and laws as written, not as he would have written them. All nine of the current justices, by contrast, have imposed "constitutional" rules-on issues ranging from abortion rights to states’ rights-that are difficult to connect to anything in the Constitution itself.

A little-known fact: White favored liberal access to abortion as a matter of social policy. Several law clerks said he told them late in his career that "if he had been a legislator, he would have been pro-choice," according to Dennis J. Hutchinson’s fine 1998 biography, The Man Who Once Was Whizzer White. Yet he could see no hint of a right to choose abortion in the Constitution.

At the time of Roe v. Wade, most leading constitutional scholars shared both views. This might seem surprising to people in whom popular culture and law professors have inculcated the notion that if you feel strongly that you should be free to have an abortion-or to teach religion in public schools, or to prohibit prayer at high school graduations, or to carry a concealed handgun, or whatever-there must be a constitutional right to do so. But White was so convinced that the Constitution had nothing to say about abortion that his 1973 dissent denounced Roe as "an improvident and extravagant exercise" of "raw judicial power."

Years later, even after the rise of feminist jurisprudence had spawned thousands of pages of law review articles praising or justifying Roe, White called it the only decision during his tenure to be so utterly without support in the Constitution as to be illegitimate. It would be hard today to find a law professor personally favoring abortion rights who would dare to doubt Roe in any public forum. The times, and the dictates of scholarly orthodoxy, changed. White did not.

Perhaps the purity of his adherence to judicial restraint reflected a lack of compassion for women desperate to end unwanted pregnancies and for other downtrodden people. This would help explain the callousness of some of White’s language, such as his suggestion in the Roe dissent that some women have abortions for mere convenience or "for no reason at all," and his assertion in Bowers v. Hardwick, in 1986, that only a "facetious" interpretation of the Constitution could support a curb on state power to punish consensual homosexual acts. Hardwick, White’s best-known opinion, struck even some conservatives (and me) as too unbending in its insistence on sticking to the original intent of the 14th Amendment. But at least White was consistent.

Not perfectly consistent, to be sure. White joined with liberals such as William J. Brennan Jr. and Thurgood Marshall in expanding judicial power as much as necessary to enforce civil rights, including ordering local and state governments to use controversial and disruptive remedies such as forced busing to desegregate public schools. Indeed, in Missouri v. Jenkins, in 1990, White’s unprecedented opinion for the 5-4 majority upheld a lower-court decision ordering a local government to impose a tax increase to finance a costly desegregation plan. White had seen American apartheid close up when, as Robert F. Kennedy’s deputy attorney general, he had tangled with segregationist Southern officials. Never has there been so weighty a reason to stretch judicial power as the need to dismantle that apartheid system.

In later years, White sometimes parted company with civil-rights groups, and sided with the Court’s conservatives, by striking down some state, local, and private racial and gender preferences. Was "the ideal New Frontier justice," as John F. Kennedy had called him, becoming more conservative? A bit, perhaps. But White (like JFK) had never been liberal in today’s sense of that word. And some of his most trenchant dissents from liberal decisions came early in his tenure, including his excoriation of Chief Justice Earl Warren’s 1966 creation, in Miranda v. Arizona, of the famous rules requiring police to tell arrested suspects of their rights to remain silent and to request lawyers.

Twenty-four years later, in 1990, White surprised journalists who had hung the "conservative" label on him by providing the fifth vote to uphold the Federal Communication Commission’s congressionally ratified racial preferences for blacks and other groups in competing for certain broadcast licenses (Metro Broadcasting v. FCC). It’s hard to believe that White viewed the FCC’s preferences, which went mostly to millionaires, with anything but distaste. And he had not suddenly become liberal. Rather, his vote was consistent with his career-long pattern of deferring to Congress, which in his view had far broader powers than the states, especially in matters of race.

This pattern contrasts starkly with the determined push by today’s five more-conservative justices to expand both states’ rights and the Court’s own power at the expense of Congress. White would almost certainly have sided with the more-liberal justices in opposing many or all of these decisions, some of which are very hard to square with conservatives’ claims to be apostles of judicial restraint. Those are the decisions since 1996 that bar private damage lawsuits against states for violating various acts of Congress, even when the statutes explicitly authorize such lawsuits.

Last year, for example, in University of Alabama v. Garrett, the justices struck down a provision of the Americans With Disabilities Act authorizing damage suits for suspected discrimination by states against disabled state employees. Chief Justice William H. Rehnquist ruled for the majority that emanations and penumbras from the Constitution’s structure and original meaning guaranteed the states a sovereign immunity nowhere to be found in the text of the Constitution itself-a mode of reasoning a bit reminiscent of Roe v. Wade. And while acknowledging the clear power of Congress (under Section 5 of the 14th Amendment) to pierce that immunity if the states had discriminated pervasively against disabled people, Rehnquist dismissed Congress’s detailed findings that states had done just that. He called the findings and the evidence amassed by Congress "minimal," "anecdotal," and unproven.

Had White still been on the Court, he might have pointed out that this decision not only overruled the people’s elected representatives but also disregarded the text of the Constitution. It also abandoned the Court’s traditional deference to congressional findings on complex issues of fact and (like other decisions since 1996) broke with precedents from 1989 and before. White might also have noted that-absent stronger historical evidence than the Court has ever mustered that the Framers had implicitly recognized an inherent guarantee of state sovereign immunity unmentioned in the Constitution itself-Garrett seems impossible to reconcile with any principled definition of judicial restraint.

To be fair, the current wave of conservative judicial activism was a predictable (if regrettable) response to the Court’s legacy of liberal judicial activism under Chief Justices Earl Warren and Warren E. Burger. Conservative justices who have inherited precedents bending the Constitution to serve liberal political ends are sometimes justified in seeking to unbend it, by overruling those precedents. And when they see their liberal colleagues putting points on the board in the culture war by making up constitutional law for the sake of causes such as abortion rights, conservatives may understandably (if unjustifiably) be tempted to do the same for the sake of causes such as states’ rights, when they have the votes. Thus does activism beget activism.

"Judges have an exaggerated view of their role in our polity," as White once remarked. And with the passing of old-fashioned jurists such as White and John Marshall Harlan Jr., who saw themselves as umpires, not philosopher-kings, that exaggerated view has become a self-perpetuating phenomenon.