It's not just cooling...

[hwg]

I saw a reference to this site on Digg. Very interesting projects. It is impressive what you guys have achieved.

I know cooling is extremely important in overclocking, but I didn't see reference here to anything else. When you get up to 5,6, 7 GHz and beyond, you might be hitting limits that are not thermal, but related to signal integrity, timing, and propagation delay inside the CPU.

A 5GHz clock has a period of 200 picoseconds. Electrical signals in a chip travel at a rate of about 100 picoseconds per inch. So in that clock cycle, signals are traveling about 2 inches.

For the CPU to work correctly, internal clocks and other signals need to arrive at the right time, but they don't all travel the same path, or the same length of path, so you have skew (difference in time between two signals arriving). Also, the other signals (address, data, and control signals) need to be stable some time before the clock (setup time) and some time after the clock (hold time). Since you need some time before and after each clock edge, that limits how short each clock cycle can be. (For example, if minimum setup is 100 picoseconds and minimum hold is 100 picoseconds, the clock cycle has to be at least 200 picoseconds, which is 5GHz). No matter how well you cool it, the chip just won't run any faster.

All chips are designed with some margin on timing. That's why overclocking works at all. But there is a limit. It varies slightly from one chip to the next, just due to manufacturing tolerances. So, even with perfect cooling, you might max out at 1.8X overclocking on one chip and get 2X on the next one of the same part number, from the same batch.

The guys who have achieved 8GHz certainly have extreme cooling, but they also "lucked out" and got a CPU with really good timing margins.

I didn't see any references here to these kinds of electrical issues, so I just thought I'd point it out. Apologies if it has been discussed before.

Just out of curiosity, has anyone tied just dunking the whole motherboard in liquid nitrogen? I don't think LN2 conducts electricity, so that should work.

Cheers, and good luck in all your efforts.
hwg

[V2-V3]

Originally Posted by hwg

I saw a reference to this site on Digg. Very interesting projects. It is impressive what you guys have achieved.

I know cooling is extremely important in overclocking, but I didn't see reference here to anything else. When you get up to 5,6, 7 GHz and beyond, you might be hitting limits that are not thermal, but related to signal integrity, timing, and propagation delay inside the CPU.

A 5GHz clock has a period of 200 picoseconds. Electrical signals in a chip travel at a rate of about 100 picoseconds per inch. So in that clock cycle, signals are traveling about 2 inches.

For the CPU to work correctly, internal clocks and other signals need to arrive at the right time, but they don't all travel the same path, or the same length of path, so you have skew (difference in time between two signals arriving). Also, the other signals (address, data, and control signals) need to be stable some time before the clock (setup time) and some time after the clock (hold time). Since you need some time before and after each clock edge, that limits how short each clock cycle can be. (For example, if minimum setup is 100 picoseconds and minimum hold is 100 picoseconds, the clock cycle has to be at least 200 picoseconds, which is 5GHz). No matter how well you cool it, the chip just won't run any faster.

All chips are designed with some margin on timing. That's why overclocking works at all. But there is a limit. It varies slightly from one chip to the next, just due to manufacturing tolerances. So, even with perfect cooling, you might max out at 1.8X overclocking on one chip and get 2X on the next one of the same part number, from the same batch.

The guys who have achieved 8GHz certainly have extreme cooling, but they also "lucked out" and got a CPU with really good timing margins.

I didn't see any references here to these kinds of electrical issues, so I just thought I'd point it out. Apologies if it has been discussed before.

Just out of curiosity, has anyone tied just dunking the whole motherboard in liquid nitrogen? I don't think LN2 conducts electricity, so that should work.

Cheers, and good luck in all your efforts.
hwg

Hey thank you for the info, there is a lot of valuable information for us. but the problem like you said is finding a good CPU, we dot exactly have the ability to change timings in the CPU to account for mis matched latency.
but i will look into this matter.

submerging a motherboard in LN2 is a good idea as the standard heat sinks would work perfect and no insulation is required. however CPU's are "Cold Bugged" so most processors will not even operate under -170C
Core 2 chips are known to stop working around -150C
and MOSFETs also fail when below -150C so regulation of temperature is key.

you have to have the right balance of temperature, voltage, frequency, and luck to Overclock a CPU on LN2.

[dekard]

hwg, have you thought about working that into a wiki article?

[hwg]

If I have a bit of time I could maybe do that. However, I'm far from an expert on overclocking. Just an EE who designs circuit boards and knows some of the general issues.

To expand a bit on my previous post, all chips are designed to work at some nominal temperature. Of course they work over a wide range of temps, but they are designed for best performance at a certain temp. Now, I don't design microprocessors, but I do use them in my designs. My guess is that the chip designers are trying for best timing margins at the temperatures the chip is most likely to see. Let's say the average CPU is going to be in a box at room temp (20 to 25C). You have a heat sink and fan, and the chip is generating heat, so its internal temperature is going to be higher, lets say 50C, just as an example. So the chip designer is going to design for max timing margins at 50C.

Now, when you overclock it, of course you have to dump out the extra heat you generate from running it faster, so that the chip does not overheat. So you use more exotic cooling. But I'm not sure you want to cool it TOO well. (My comment about dunking the thing in LN2 was kind of tounge-in-cheek...) If the chip is designed for best timing margins at an internal temp of 50C, and your fabulous cooling has it running at an internal temp of -50C, the chip just might hit the limit on its internal timing sooner than it would at 50C. My basic point is that colder is not always better. It is usually better up to a point, but beyond a certain point, you might be making things worse by going colder.

I have personally seen chips (not CPU's, but other digital devices) that work fine at room temp, but start to show flaky behavior at -5C. And that's -5C ambient, so the chip internal temp was probably above 0.

Generally speaking, semiconductors do better at cooler temps. They use less power and are able to run a little faster. But that only goes so far. If you take it too far out of its designed temperature range, it might get worse, especially at the hairy edge of 5GHz and beyond, where the timing margins are miniscule to begin with.

If I was going to dive into overclocking, I would probably do an experiment, where I used something like a TEC (thermo-electric cooler) with a control loop around it, to control the chip temperature as accurately as possible. Then I would do a bunch of test runs at different temperatures and plot Fmax vs. Temperature (the internal CPU temp). I would expect to see Fmax (the maximum CPU speed) peak somewhere in the neighborhood of 0 to 20C, and get worse on both the hot and cold ends. I've never done anything like that, so that's a pure guess. If anyone here has done that experiment, I'd be interested in seeing the results.

Once again, I'm no expert in overclocking. You guys are doing some very cool stuff (no pun intended). Good luck on all your projects.

hwg

[V2-V3]

Vey informative, thank you!

we have been well aware of how temperature effects latencies, but we have had no way to get measurements of these timings. for us its a big hole in our testing that needs to be filled. we became aware of the effect of temperature on the internal timings when the CPU/RAM/North Bridge operated at drasticly different temperatures. mainly the signals were not being arrived as anticipated (we believe) and the system failed to operate, Overclocking with LN2 requires and acquired skill of temperature regulation. and when we are going for 8Ghz we have found a balance of temperature and voltage are key to Frequency. the latencies do hold us back quite a bit.

but we do require such low temperatures to allow the Transistors to operate with ~2 Volts otherwise we could not stabilize such a high frequency.

do you know of any way to modify these timings or a way to find specifications for the CPU tolerances? i have asked several Intel engineers who have no idea and believe that such information is keept secret.

[hwg]

I doubt that you would find those kind of specs published. And there's no way I know of to modify or tweak those internal timing parameters. They are what they are, and you live with it. You are right, the only things you have control of are temperature and voltage.

I'm starting to understand where you're coming from with the low temp, low voltage. (I told you I'm new to this...) Low temp gives you less thermal noise (background noise that all semiconductors produce). Lower thermal noise allows you to use lower voltages, since there is less error on the switching threshold. And lower voltages allow the chip to run faster, since the switching time of a transistor is limited by its slew rate, or dv/dt. For instance, if the transistor can change at 10V/ns, and it only has to change by 2V (0V to 2V), then it can switch in 0.2ns. If it has to change by 3V, it will take 0.3ns, since the rate of change is the same (roughly). So all those things are working for you as you go colder. But the one thing that may be working against you is the one you can't see or control, and that is the internal timing. It may be optimized for a much higher temperature, and it might get worse as you go colder. But there's nothing you can do about that. You just have to find the the optimum temperature where things work best. My only point is that "optimum" might not equal "as cold as you can make it". Maybe it is.

Your comment about "stabilizing such a high frequency" has me thinking. The clock that is fed into the CPU is not 8GHz, but a much lower frequency. The CPU has a PLL (phase locked loop), or something similar inside to generate the internal higher-rate clock. A PLL is basically a feedback circuit with a phase detector, charge pump, and a VCO (voltage controlled oscillator). These things are all in silicon. But there are usually a couple of external components, a resistor and a capacitor, which make up the "loop filter", and set the natural frequency range of the PLL. Sometimes there is only a capacitor, as the resistor is on-chip. Sometimes there are no external loop filter components at all. If the CPU you are trying to overclock has an external loop filter, I wonder if it might make it easier to stabilize the clock by changing the component values in the loop filter (different C and/or R). The PLL will be designed to run at the CPU's normal clock rate. Maybe you could tweak it to shift the frequency up so that the clock will more easily stabilize at higher frequencies. You will still ultimately be limited by how fast the VCO will run, internal timing, etc., but at least the PLL would be more optimized for higher frequency operation.

If I can find a data sheet for an Intel CPU, I will have a look at this and see what I can find out. Or maybe you guys have already looked into this.


hwg

[blue_fire]

I have to agree with hwg on this one. I don't think there is any way to modify latencies on die without doing it at the manufacturing lvl.

hwg you do mean lower temps, higher voltages right? Less noise allows for more voltage, because it causes less noise at these lower temps due to greater efficiency at the electron lvl. And higher voltages allow faster speeds.

There is a couple of things i think need to be corrected but i may be wrong. CPUs are usually not cold bugged on the intel side. Most of the time it's the north bridge that start producing errors. RAM operates much slower the CPU (except for the AMD side of this) and the north bridge has to regulate this timing difference in between the too. Ours kept failing at ~ -30c, which is about were amd processors fail at too. P35 and X38 should help that due to better man fab but who knows?

A little while back i was reading reading some neat stuff aboutP965, 680i and 975x "straps". The north bridge has a set of timings of its own, BUT it has one diffrence: The ability to change the memory timings (on the northbridge) with the fsb. Yielding better performance. A lot of people were running 1333 with a 1066 timing and it seemed to help.

This may have kind of a random structure, but my post leads to one thought.
As the latencies in a CPU are set and optimized, the may have to slow down or speed up as an attempt to keep up or slow down with the rest of the system. As the CPU get's colder, the latencies and timing may be able to slow down or speed up as needed. I don't think this applies to the physical layer, but those get faster as they get colder too.

I believe that 8GHZ was not only achieved with a lucky processor with good timings and latencies, but with components that tolerated the lower temperatures better.

EDIT: if anything doesn't make sense ask me to clarify, my thought process is a little out of order and slightly random.

EDIT EDIT:
"When you overclock the FSB, the internal NB frequency also increases (logically). The NB "Strap" is used to loosen the internal timings/frequency(?) of the northbridge to ensure it's still operating within specified levels. (i.e running stabley). The strap changes loosen the internal NB timings to allow the FSB to be pushed further (Hence supporting 1066/1333 FSB chips without pushing the NB into instable territory). The trade off is, at the strap change point (~400 FSB on the P5B for instance) you get a performance drop as the internal latencies are much higher. The performance difference is negated however as the FSB can be pushed further thanks to the looser internal latencies. "
Borrowed this from Hildand3r bit-tech.net Forums - Ram question for aw9d max

Cooling helps keep these timings tighter. Which is why NB cooling is always somthing i have stressed. DI should be used for the NB imo

And this
article is why im getting the P5K

[Backburn]

First off, excellent information hwg! It is refreshing to find someone who has a grasp on CPU architecture and electrical engineering to boot.

Any ideas or help you can provide would be awesome; and welcome to the forums!

I have to agree with hwg on this one. I don't think there is any way to modify latencies on die without doing it at the manufacturing lvl.

This is true. The trace lengths in the proc dictate timing. Unfortunately, we don't have a fab at our disposal.

HOWEVER, if we able to get a custom proc / mobo you could do several things to keep the timing issue at bay.

CPU
1) Disable all xtra instruction sets. sse, mmx, etc..
2) Disable HT.
3) Disable extra cores.
4) Nerf the cache to like 32k. (We are just going for frequency right?)

Starting to sound like a Celeron now


Motherboard:
1) Kill the ISA bus
2) Kill the PCI bus.
3) Hell, get rid of everything.

If we could get a company, say ASUS, to produce a P5B Deluxe with no sound, raid, NIC, etc... That would be the most viable solution.


I've listed a few things that would help, but there are a ton more. Let's make a list. Any ideas?

[blue_fire]

First chance of me getting money, ill have a P5K. Been reading a a lot about bearlake.