June 17, 1994 | Given Jack Nicholson's predilection for leaping over the top and feasting on the scenery, the restraint of his performance in Mike Nichols' Wolf is pretty close to astonishing. Indeed, it's probably the most subtle piece of acting Nicholson has done since his brilliant cameo as Eugene O'Neill in Reds. And what makes it all so delightfully surprising is, subtlety is the last thing you would expect in anybody's screen portrayal of a man who turns into a beast after being bitten by a wolf.

But, then again, Wolf itself is hardly your standard-issue werewolf melodrama. In fact, the word ''werewolf'' is never spoken in the movie.

Written by Wesley Strick and novelist Jim Harrison, this is a film of cunning wit, nimble intelligence and uncommon persuasiveness. Under Nichols' calculated and graceful direction, it is slick, sexy and scary in meticulously measured increments. What it isn't is a film filled with cheap shocks and unfettered bloodletting. Wolf is a serious and largely successful attempt to find the heart of darkness that lurks beneath the surface of B-movie hokum. To accomplish this, Nichols has gambled heavily on his ability to suspend audience disbelief while taking the frankly fantastic and placing it within a vividly realistic context.

The opening scene sets the tone. It is, of course, a night as bright as only a full moon can make it. Will Randall (Nicholson), a Manhattan publishing executive, is driving along a snowy Vermont road, on his way home from a business meeting, when he hits a wolf with his car. Thinking the animal is dead, he tries to drag it off the road -- and gets nipped by the still-living beast before it dashes off into the dark woods.

Back in Manhattan, Randall is caught in a maelstrom of midlife crises. He fears the worst from his publishing company's new owner, Raymond Alden (Christopher Plummer), a suavely crass billionaire, and his worst fears are entirely justified. Alden turns Randall's position over to our hero's devious protégé, Stewart (James Spader, once again hugely convincing as a yuppie from hell), and offers Randall the consolation prize of ''a choice between no job and a job no one would want.'' So much for the bad news. The worse news: Charlotte (Kate Nelligan), Randall's wife, is having an affair. The worst news: Stewart is her adulterous lover.

Under normal circumstances, all these setbacks would likely leave the self-deprecating Randall crushed and defeated. But Randall feels too good to let any of this get him down. He feels younger, healthier, happier. His sensory powers have drastically increased -- so much so, he can hear conversations from across crowded rooms, and uncover his wife's unfaithfulness by literally sniffing out her lover. His personality undergoes an equally dramatic transformation:  He develops a strain of steel-trap ruthlessness that enables him to regain his old job, humiliate Stewart, and even impress Alden.

More important, he even manages to capture the attention of Alden's beautiful daughter, Laura (Michelle Pfeiffer), a jaded voluptuary with heretofore untapped reserves of strength and passion.

Trouble is, there's a downside to all this: With each new full moon, Randall becomes a bit more beastly. He sprouts unsightly hair. His teeth grow larger. And it may not be long before it takes more than a deer to satisfy his mounting bloodlust.

Nichols has wisely toned down the special effects in Wolf, allowing make-up wizard Rick Baker (An American Werewolf in London) to make only minimal enhancements of Nicholson's natural animal magnetism. That's altogether appropriate for a movie that isn't so much a monster melodrama as a thoughtful metaphor for the quandary facing anyone of taste, culture and moral standards in a decadent society that places little value on such luxuries. Randall, perhaps the last honest man in contemporary publishing, must turn into a beast to protect his turf, and himself, from the vandals, betrayers and philistines around him. It's a clever concept, one that triggers the sharpest dialogue in the movie's funniest scenes. But Wolf doesn't play the concept entirely for laughs -- Randall's loosening grip on his own humanity is never treated as anything other than cause for alarm.

Ironically, Wolf is least successful when it tries to deliver old-fashioned B-movie thrills and chills. Nichols has an unfortunate fondness for slow-motion cinematography during the wolf rampages, particularly during a climactic battle between Randall and his most formidable foe. This sort of thing looked a lot better on such 1970s TV series as The Incredible Hulk and The Six Million Dollar Man.

Wolf may be too cerebral for its own good, or at least too smart to get away with dumb fun. But the film is exceptionally well-acted, particularly by Nicholson and Pfeiffer, and often casts a darkly romantic spell. Better still, it remains dramatically involving and emotionally compelling in ways that go far beyond the simple expedient of jumping up and yelling ''Boo!''