Please submit ideas or thoughts about tools, techniques and resources you love or hate.
Charley Moore has an update on Beyond Compare:
Since a lot of readers mentioned using Beyond Compare, I thought that they might want to knew that they now have a version 4.0 in Beta. It includes an OS X version (finally!) , better picture and hex compare, SVN and Dropbox integration, plus more customization of unimportant changes. Disclosure: I have no association with Scooter Software, other than loving BC and recommending it to every developer I know.
INVESTINTECH sent me a copy of their Able2Extract Professional 8 software. It's Raison d'être is to convert .PDF files into other formats. There are versions for Windows, Linux, and the Mac. It's easy to use, and has unusually-well done tool tips that lead you through the conversion process. The program can convert to these formats:
Nicely, it will do batch conversions as well; aim it at a directory of files and walk away while it does its work.
Convert a typical .PDF sales brochure to, say, Word format and it can even keep the original document's images and much of the formatting, though don't expect anything near a mirror image of the original document. But now the text is editable. There's a text-only mode if you don't want the pictures. It will OCR as well, so those scanned .PDFs can come back to life with the text being editable.
Muse 254 included an image of coefficients for smoothing data. I converted that Muse to .PDF and had Able2Extract convert the coefficients to text. No luck; it seemed confused by the associated prose and assumed that the image was, well, an image. So I printed it out, scanned it, and then Able2Extract perfectly OCRed all of the coefficients into an Excel document with them formatted just like the picture, except now they were actual numbers in cells. Pretty cool. It's generally very accurate in doing conversions, but does get tripped up from time to time and generates meaningless garbage. OCR is hard, and I have yet to see a program that is flawless. All of the scanned C code I tried was converted perfectly.
As a .PDF reader it's so slow as to be useless. We're talking about a whole new world of slowness. OCRing is fast on small documents (a few pages) but rapidly slows. A 1000 page poorly-scanned document took about 90 minutes.
For $130 it's a decent deal if you need to do a lot of OCRing. There is a seven day trial so you can see if it meets your needs.
Gonzalo Sanchez had some interesting information about the filter I wrote about last issue:
The use of Savitzky-Golay smoothing polynomial filters is mentioned in the - I believe, well known -'Numerical Recipes in Fortran/C/C++' books (http://www.nr.com/). Interested readers can access the older versions of the book online, as well as a limited number of pages of the current version.
The section on S-G filters (section 14.9, starting at p.766 on the Third Edition of N.R. in C++) shows some plots that compare the results of filtering simulated noisy data with a simple moving-average filter and a Savitzky-Golay one ( Fig. 14.9.1, p.771).
I recall me and a classmate playing around with Matlab and these filtering polynomials time ago, to smooth data from digitized signals pertaining to radar echoes. At least then, a correlation operation (which can be viewed as yet another FIR filtering operation) was typically used to maximise the probability of finding an radar echo buried on the received signal's noise---but that just gives the best position of the echo at the expense of destroying other information such as pulse height and, particularly, width.
When we saw the section on S-G filters in the NR books, we gave these a try and, at least qualitatively, they definitely did a superb job. In fact, the cited book says these filters are well suited for pulse processing, and we believed that they are particularly so if you know or can estimate the height and width of the expected pulses. (A word of caution, though: we learnt that noise in radar echoes is not Gaussian, due to the rectifying process at the receiver; we did not pursue the issue further at the time, so I cannot tell if this 'non-normal' p.d.f. for the noise could make a significant difference thereafter).
If my memory does not betray me, we obtained best results in terms of recovering the FWHM (full width at half of maximum) of the pulses' echoes playing with a quadratic polynomial (not a quartic one) and roughly the same number of coefficients for the S-G filters as number of samples expected in a 'perfect', noise-free, radar pulse echo (the latter figure, estimated from the digitizer's sample rate and radar transmitter's specs).
Examples of the effects of varying filter widths on simulated signals with added noise are plotted in , Fig.14.9.2, p.772.
 "Numerical Recipes - The Art of Scientific Computing" (3rd. Ed); W.Press, S.Teukolsky, W.Vetterling, B.Flannery. (C) C.U.P. 2007.
Jon Titus wrote:
Good to see mention of Savitzky and Golay in your newsletter. I used their techniques in grad school to help analyze data from a fast-scan spectrophotometer. I wanted to let you and your readers know about updated information and a few corrections to the original article. People interested can refer to K. R. Betty and G. Horlick, Analytical Chemistry, volume 49, 1977, page 351. I used to have those articles in a file, but either I can't remember which file, or I have tossed them.
Unsurprisingly, a number of people responded to my comments last issue about the need for a degree. Paul Bennett wrote:
In the UK the modern apprenticeship takes youngsters and gives them practical on-the-job training coupled with one day per week at college for the theoretical side. They do not have to make the choice between the practical work or the theoretical and, with appropriate efforts on their part to do well, will get the chance to progress to doing a degree with one of the Universities (all paid for by the employer). This would seem like the best of both worlds, especially when they come out with all the practical knowledge and a degree. I would certainly encourage any of our youngsters here to have a serious look at the Modern Apprenticeship route.
Additional to that the UK has also started a number of University Technical Colleges which take in pupils from age 14 through to 19 in a training environment that has a degree of practical training in engineering and science and starts their Engineering Academic training a couple of years earlier. When the world needs many more engineers all countries should be looking at such means to expand the pool of available, worthwhile, technical talent.
Dave Telling contributed his thoughts:
Not being able to integrate a function -- I took college calculus at an advanced age -- in my forties, as I wanted to see how well I would do, as I was not a college graduate. I ended up with a 4.0 GPA, and a comment by one of the instructors that I should consider continuing in my math education studies. Since then, I have had to do essentially zero calculus -- virtually all of the formulae that I use in my designs are "canned", plug-in-the-numbers formulae. I could not, even if my life depended upon it, integrate or differentiate a function with any degree of confidence now. I found out that "what you don't use, you lose" is very true with a variety of technical subjects. On the other hand, I still remember a whole lot of stuff about vacuum tubes, because I spent hours and hours on tube circuits as a youngster. Second -- we hired a young kid with a BSEE into our engineering department back in the late 80's. I was assigned to "mentor" him regarding circuit design and operation, especially analog circuits. At one point, I was discussing a transistor circuit, and I said something like, "Looking at the base, you should expect to see about .6 to .7 volts when the transistor is biased on." He looked at me and replied, "How do you know that?". I, in turn, replied that it was a very typical voltage for an NPN silicon transistor under normal operating conditions. He asked again, "Well, how did you know that?", to which I replied, "Didn't you study transistor circuits in school?" and he said, "Yeah, I remember them talking about something like that, but I don't really recall very much."
Once again, because the educational process was designed to stuff information in, then regurgitate it during the test, with no time spent in practical application, he had completely lost his knowledge when out in the "real world". He eventually quit and went to work for the local telephone company.
In my years of working with technical people, it seems that the best were those who enjoyed electronics as a hobby, and designed/built projects on their own that had to be troubleshot and repaired. Those who were in it for the money, or job security, or some other reason just didn't have the "spark" or "knack" to do well, especially in analog design.
A strong interest in, and an enjoyment of almost any activity will result in competence, because you want to do it.
Bruce Wedding agreed with my advice to wannabees:
Having spent the last 25 years working in software development without a degree, I make the same recommendation you do, to anyone asking how to break into our world. Go to college.
I've been lucky enough to work as a well-paid software engineer for several Fortune 500 companies. I've written code for Class III FDA approved medical devices. I've designed and developed deep-sea drilling, blow-out prevention control systems. I've successfully managed more than a dozen engineers. But every time I'm faced with the prospect of seeking employment, I feel that familiar knot in my stomach.
Right or wrong, so many doors are closed to me. Regardless of my past positions and accomplishments, at many companies my resume moves from HR straight to File 13. And while I've always done well and never had trouble finding suitable employment with equivalent pay, the thought that, "I got lucky one more time" is ever present.
There is one benefit though. Recruiters stop phone harassing you as soon as you let them know you don't have a degree :)
The bottom line is, life is much easier with the degree and I'd never recommend any other way.
David Bley wrote:
I don't have an engineering degree. I have an AA in General Studies. I have taken most of the core courses in Electronics Technology and I made it to my senior year in Industrial Arts Secondary Education. I have been an electronics hobbyist since I have been quite young. I started out working as an Electronics technician in my junior year of High School (I took three years of electronics in High School that was Jr. College equivalent) doing service work and then continued doing service and then moved into quality where I ended up doing circuit design and manufacturing. I continued my upward path and ended up spending about 30 years designing electronic medical equipment and helping others learn to design. After our business was purchased our plant was closed and relocated elsewhere. I find now, with no degree and close to retirement age, I am unable to get a job. Every place that I have been hired, I have moved on up and whenever I left I was told that should I want to come back that there was a job for me.
I had a unique approach to design and I got the tough projects that others were not able to complete.
From my experience I would recommend that anyone who wants to be an engineer should get a EE but get as much practical experience as possible. I also recommend that the student keep in mind that the field is changing so rapidly that continuing education is necessary and plan to have an alternate career around age 50.
Fabrizio Bernardini had a metaphor to battlefield promotions:
I am once more impressed by your thoughts and by the fact that across the ocean I often feel the same way as you on this topic. Actually I could have written the same thing using different examples.
As I am working with engineering students I try to pass them the fact that not everybody is born to be an engineer. And engineering requires as much as a vocation as medical school or other "high level" professions. It comes from some inexplicable internal urge to make the world a bit better using technology. This urge can make an engineer from someone without a degree (and there are examples in other branches of engineering) but it is difficult that without that you can be a good engineer, versus for instance an engineer-employee, someone versed in technicalities but not a creative person.
Dealing also with aerospace students I am pushing them to know more electronics AND software as both are still considered side things in their profession, while that actually depends entirely on software and electronics. Here in Italy students from that area are very seldom happy with their job hunting and most of them revert to other professions, if they acquired additional skills.
And skills is a magic word: is something additional you have to acquire during you university years because later it will become more difficult (time limitations). I try to push them to some technical hobbies, programming, small embedded systems into their hobbies, mix and match specialties. The engineering curricula is not enough to make an engineer, to seed and grow that "urge" that not everybody is born with.
Technicians: regrettably there are still technicians whose skills as engineers are not recognized (but the number is decreasing, and there is equivalent increase in the good engineers field). There should be a way to confer honors degrees to them, after years of proven practice. Something like being promoted in the battlefield.
I also met exactly persons from the literature environment that, without jobs, gave a try to programming and they were my best students.
I agree with you that going for a degree is a must today. But with the understanding that the piece pf paper they will give you at the end is not the proof of your engineering skills, but just a license to learn.
Zack Mully wrote:
I always look forward to your emails, and your piece on "Is a Degree Necessary?" struck a chord with me.
I spent the last 14 years focused on building digital media technology. In that time I easily interviewed over a hundred candidates for technical positions. A truth I learned from that experience was the degree had little correlation with the technical abilities. Perhaps this is different in the world of EEs and the like, but for software development, many of my best staff had no formal education in the field and usually outclassed those who did. Why? They were better, in my opinion, at scoping and stating the problem. Perhaps because they felt inadequate without a degree? Perhaps because their backgrounds gave them better analytic capabilities? I don't know.
On the other hand, I was always horrified by how poorly prepared newly minted CS grads were for the real world. Even if they'd spent the past 4 years getting drilled with fundamentals and theory, the large majority of them were hard pressed to tell me how to apply that knowledge to the problem at hand, much less express themselves clearly and persuasively.
And while I agree with you that there are no shortcuts, I remain unconvinced that degrees are the best proxies for it. Part of the problem, as I've experienced it, is that everyone wants finding talent to be cheap and easy, and defaulting to using degrees is a fast and dirty way of doing it. HR doesn't want to/can't do the leg work to get in talent, so instead defaults to keyword filters. Hiring managers are constantly fighting to get positions budgeted, so don't take chances on someone without the right degree. And the employees themselves discriminate against those without their same backgrounds. It's an openly hostile environment for those of us who are bright, well rounded and have no stomach for management positions, but don't have the degreed technical chops to get even a junior position. Not to mention that the embedded field, in comparison to the software development field and to an outsider like myself, appears to be closeted, stuffy, and not particular progressive or welcoming.
If MOOCs hold hope for you, then the maker movement holds it for me. I think the rapid democratization and price reduction of the technology is causing fundamental shifts in the field and will benefit everyone from the noob with an Arduino to the senior EEs at the big semiconductor companies. These are the people working to change the perception of tech, to get kids excited about it, to make the field welcoming to females, and to give everyone a fair shake regardless of their degree. The time and energy they're spending making exciting things and engaging people with the technology will pay dividends to field regardless of degrees issued. Wouldn't you love to see a future where people can easily become native with the technology from an early age? Where it's just as likely the English major has just as much experience with the technology as the EE?
If the established ignore what is going on in the hackerspaces and bedroom labs simply because those people don't have the right degrees, they do so at their peril. It's a classic setup for disruptive market behavior.
Personally, I have several degrees (a hard science and a foreign language), but going back to school for EE is probably out of the question for me, both financially and philosophically. I would kill for opportunities to work and learn alongside smart engineers, but they're not out there. Instead, I'm considering starting my own business so that I can counter the criticisms of not having an EE degree with my research and products. And while I'd love to apprentice myself, the DC metro region has lack of small, risk taking business. It's honestly making me consider moving to another area.
After proof-reading this over a couple of times, I realize I can sound like I'm belittling the field, whereas I am actually really impressed (and somewhat envious) by the level of dedication that most have to the profession. I just wish a smoother gradient of talent and experience was more widely accepted.
Finally, Chris Gammell, who is also doing some interesting on-line electronics education via his "Contextual Electronics," wrote:
I recently found an article about "hacking your education" (and a second one doing the same, except with MOOCs). These are all about getting a degree as fast and cheaply as possible, ostensibly to get past the gatekeepers and show off the practical knowledge you already have during a job interview. At least that's how I viewed it. It might not work if the job requirement is written as needing "an ABET accredited engineering degree" (which many do), though perhaps this method could be used as a workaround. Still, I think it's an interesting exercise when the degree is treated as a box to be checked and the ability of the candidate to do the job is the true focus.