Its going to depend a lot on luck of the draw. Especially considering you are looking at a imported rifle that has lived a few decades under someone elses care.
If both rifles were new in the box, I would probably go with the Annie. Just for the simple fact its very easy to sell a Annie sporter in my area if I was not satisfied with how it shot. But considering the facts that its going to be almost impossible to find a New 164, I would recommend the CZ.
I have a S/A 141, which is virtually the same rifle as the 164, but with the upgraded stock, that looks a lot like the Weatherby stock. The rifle looked ment on the outside when I bought it, but from day one it would never shoot. I tried everything in my power. I considered re barreling it, but never went through. When I got the barrel estimate the gunsmith scoped the barrel. I was amazed to see the barrel was badly pitted and eatn up. The S/A 141 is Ok for small game hunting, but its not the best on paper.
Of the two, I feel the S/A has a better trigger after its been adjusted down (But the CZ can still give you a decent trigger for punching paper). But I feel bolt design of the CZ is a little better, if you can live with that crazy safety.
So in turn my CZ shoots rings around my SA141. But at the same time I have had a fair amount of 64 actions out shoot my CZ. I have two 54 action sporters (1422 and a 1712) and both of them consistently shoot noticeably better than my CZ. But considering they are about twice the price and have MUCH nicer triggers, I expect them too.