Intermittent Fault found after WEEKS!

[awemawson]

I've been chasing an intermittent fault on my home brew CNC Plasma Table for ages, and I THINK I may have bottomed it at last. Symptoms were that after maybe half an hour of use or maybe much longer I would loose control of the steppers. Mach3 screen positions would change but not the steppers.

Now as it's electronics are in an industrial cabinet with a safety interlock it was impossible just to 'open the box' and measure stuff. Fault most often arose at the end of a job, and then the other day I noticed that a sharp tap on the cabinet would bring on the issue. OK overcoming the interlock I started running with the door open and applied the 'tap test' ie tapping in the manner of a reasonable man with an insulated screwdriver handle. I soon narrowed it down to the area of the Omron relay that is my Estop.. Could be wiring, the relay socket or possibly the relay itself.

Wiring seemed sound as a bell, socket looked fine as did the relay. Spare relay and socket ordered, which arrived today. Quick test to prove that I could still bring on the fault (I could) then install the new relay. Much tapping banging and thumping later - fault gone.

That was 6 hours ago, and I can no longer induce the fault

The offending relay:

My theory is that as the coil is powered all the time it must run a bit warm and the way it is mounted it is under the four sets of change over contacts - spring retainer not quite strong enough and vibration (or me thumping it!) was inducing contact bounce tripping the 68 volt power supply for the stepper motors.

. . .trouble is with intermittent faults . . how long before you KNOW it's cured !

 

[Misterg]

The offending relay:

View attachment 447819

That's interesting... I have the 12V version of the same relay as the E-stop on my home made CNC controller. Very occasionally, the E-stop will trip, but always resets OK. It still happened with the E-stop loop shorted out so I traced it to the circuitry around the relay. I have occasionally been able to induce it by tapping near the relay, so assumed it was a bad connection and re-worked the wiring around there without really finding anything wrong. The problem still happens very occasionally though, so it might be new relay time...

I did buy the original on ebay

 

[Kayos]

When I was repairing boilers I took the view there was no such thing as an intermittent fault, it was either faulty or working correctly

 

[Red'n'Black]

When I was repairing boilers I took the view there was no such thing as an intermittent fault, it was either faulty or working correctly

Oh the fault is in there all right, but the act of looking for it makes it go away, for a while.

A Mr Heisenberg had something to say about this, if I remember rightly...

 

[awemawson]

That's interesting... I have the 12V version of the same relay as the E-stop on my home made CNC controller. Very occasionally, the E-stop will trip, but always resets OK. It still happened with the E-stop loop shorted out so I traced it to the circuitry around the relay. I have occasionally been able to induce it by tapping near the relay, so assumed it was a bad connection and re-worked the wiring around there without really finding anything wrong. The problem still happens very occasionally though, so it might be new relay time...

I did buy the original on ebay

Well from my experience I'd say replace it ! I installed mine in January 2016, so over eight years ago and off and on I've had issues for well over a year, though I did also recently install more EMI protection in the form of ferrite clip on chokes which eliminated a fault with different symptoms.

 

[Morrisman]

When I was repairing boilers I took the view there was no such thing as an intermittent fault, it was either faulty or working correctly

So if a boiler was deemed faulty did you just replace the whole thing?

 

[Kayos]

So if a boiler was deemed faulty did you just replace the whole thing?

No, an intermittent fault is just a fault to be fixed

 

[minimutly]

You must be some infuriating person to live with @Kayos ....

I had one like this on the force draught fan (500 kW or so motor)on the powder drier at the milk factory, many years ago. The plant would run for hours, then just stop, FD fan failure. Everything was Klockner Moeller, and it was this classic change stuff in case, go again. The trouble was it would stop the production run of the drier and evaporator (one of the biggest in the UK), the losses were getting silly.
Anyway, once all the hard hats were out of the way I went through the whole control circuit, tapping, checking etc - like you @awemawson , until a tap on a simple plug in 11 pin interface relay did exactly what you describe. I don't recal if I changed the relay and came back to it, but it was actually the base that was the problem - a poxy £2.00 relay base...
In the same factory we had milk poweder bagging and palletizing lines, 24V relay logic - if you didn't get on top of it it could really fry your brain. But, after a while you got to recognise "where" the machine had stopped, so the test probes and screwdriver tap was the way to find the grain of powder dust sitting between the contacts. Tap and away to go. Truly an intermittent fault, that could be anywhere, and somewhere different every time.

 

[Robotstar5]

..... But, after a while you got to recognise "where" the machine had stopped, so the test probes and screwdriver tap was the way to find the grain of powder dust sitting between the contacts. Tap and away to go. Truly an intermittent fault, that could be anywhere, and somewhere different every time.

We had some quite complex relay logic machines at Rover and found it easier to ask the operater what the next part of the sequence was as they knew it intimately so it gave us a head start of where to look.

 

[Morrisman]

No, an intermittent fault is just a fault to be fixed

That makes it far easier to find, if you call it a fault, not an intermittent fault.

 

[gaz1]

I've been chasing an intermittent fault on my home brew CNC Plasma Table for ages, and I THINK I may have bottomed it at last. Symptoms were that after maybe half an hour of use or maybe much longer I would loose control of the steppers. Mach3 screen positions would change but not the steppers.

My theory is that as the coil is powered all the time it must run a bit warm and the way it is mounted it is under the four sets of change over contacts - spring retainer not quite strong enough and vibration (or me thumping it!) was inducing contact bounce tripping the 68 volt power supply for the stepper motors.

. . .trouble is with intermittent faults . . how long before you KNOW it's cured !

it will come back if you leave it where it is

if you think its over heating them move it somewhere cooler

as goes for vibration fit an anti vibration system in as well

 

[tigdlo]

No, an intermittent fault is just a fault to be fixed

What about an intermittently asymptomatic fault?

 

[Pete.]

What about an intermittently asymptomatic fault?

A trans-permanent fault? There'll be a law about them any day soon.

 

[Kayos]

What about an intermittently asymptomatic fault?

If there are no symptoms is it even a fault?

 

[Bullet2012]

If the fault occurs when nobody is looking and then self rectifies, did it actually occur?

 

[eddie49]

In my 44 years as a fixer of computer faults, the tough ones were the design faults. A few were mistakes in the original electronic design, and the rest were software bugs.

 

[minimutly]

Two faults interacting can be quite hard to fathom - your brain is telling you there is a fault, a single one, but if there are two, and maybe, as @eddie49 says, one is designed in, it can take some tracking down. If my logical tracing doesn't get me anywhere my next thought is there will be more than one.

 

[MetalMonkey]

When I was repairing boilers I took the view there was no such thing as an intermittent fault, it was either faulty or working correctly

Given everything will fail at some point, really they are all faulty out of the box. It's just they have periods of temporarily working for a few years.

 

[Kayos]

Given everything will fail at some point, really they are all faulty out of the box. It's just they have periods of temporarily working for a few years.

More that they follow a defined sequence of operation, so even if you are not familiar with a particular brand or model you can quickly narrow it down (they "fault" under defined conditions) then refine and repair

I have never met a boiler I couldn't fix, but sometimes the cost meant it was beyond economical repair so the customer is given a choice

 

[dobbslc]

Truly an intermittent fault, that could be anywhere, and somewhere different every time.

The ghost in the machine.

If there are no symptoms is it even a fault?

If a tree falls in the forest and no one's there, does it make a noise?