Vim & Vision

You searched for articles tagged with Rant.

[ #177 ] Meetings are Toxic Permalink

Agile, Best-Practice, Rant Added less than a year ago

[ #131 ] What is Perl? Permalink

English, Perl, Rant Added less than a year ago and last edited less than a year ago

Larry now uses "Perl" to signify the language proper and "perl" the implementation of it, i.e. the current interpreter.

Perl is a language specification. Languages (well, grammars) should be able to be expressed in BNF or EBNF or other formal notations (if you are a perl expert and have just registered alarm bells, refer to note at bottom of post.) If someone likes Perl, it might be because they like the expressiveness of the syntax. Perl can exist without any interpreter or community or libraries. Perl can exist on paper only.

perl is an application or program that runs (interprets) languages written in Perl. There is one major implementation of perl (pre-Perl 6 that is), but there is nothing stopping you from writing your own version of perl right now (except time and incredible effort, again, please see the note at bottom of post) - although you'd call it something else wouldn't you? If someone says they don't like perl, they might be talking about how the internals of the perl application are filled with deep voodoo - that the C that describes and implements perl is hard to follow and hard to extend. Or they might be complaining about the command-line flags that perl recognises. perl requires Perl to exist. If perl was interpreting a language that was not Perl, then it would not be called perl.

~Perl~ is a term I just made up. It describes the combination of Perl, perl, libraries included with the standard distribution of perl, CPAN, the Perl community, Perl books, perl books, etc. If someone says they don't like "Perl" because of something that annoys them with CPAN or perlmonks or even a library that is distributed with perl, then they are talking about ~Perl~. If CPAN didn't exist, Perl and perl could still exist!

Am I always so careful as to use the correct spelling of Perl or perl? No, I'm sure I've stuffed up a lot, including in this blog, but I've been thinking about the distinction lately so I thought I'd post on it. Perl would be useless without perl. perl would be almost useless without ~Perl~. However, if you removed all that is ~Perl~ except for Perl and perl, people would create a new ~Perl~. New libraries, new distributions, new websites etc would spring up, made possible by having a working perl. It would likely be a slightly different version of ~Perl~ than before, and that's OK.

Similarly you can like Python, hate python but love ~Python~. Actually, it turns out that there are at least three major python implementations: the so called "cpython" (the main python app, from python.org); "Iron Python" (a .NET implementation) and jython (written in Java.)

Recently, so I'm led to believe, the performance of java has improved considerably. I remember when I was at uni I would say that I hated Java because of its poor performance. I was wrong to say that, I should have said I hated java (or javac, etc.) Java is the language, and is not what I was talking about. Incidentally I don't like Java either, because I think it is too verbose and requires too much boilerplate. But that's just my opinion!

Is it ridiculous to make these distinctions? I don't think so. Annoying in polite conversation? Almost certainly. I think it's interesting to think of the difference especially with different versions of Python interpreters/compilers (see above) and Perl 6 interpreters/compilers around the place (Rakudo and Pugs for example.) You could say that you hated Pugs, loved Rakudo, loved Perl6 but were disillusioned with ~Perl6~.

Notes:

1. Do I honesty believe anyone will start using the term "~Perl~"? Of course not, I just had to type it differently for the purpose of my rant! It's even impossible to tell just by listening as to whether someone hates Perl or perl or ~Perl~ - but you might try asking them whether they are talking about the language, the executable or the community. At least that way they will be forced to back up their opinion intelligently.

2. Sometimes I see reference to PERL. PERL is a mythical creature of the past, associated with spaghetti CGI scripts from 1997, or with people who don't program much. I don't really think PERL ever existed, and if it did let's pretend it didn't.

3. Can Perl actually be expressed in BNF? Well no, it can't be expressed in BNF because it is not a context-free grammar. This article will show you some examples of very ambiguous Perl statements, and tell you that only perl can interpret them. By that it means that there is lookahead code and certain resulting decisions in perl that are not obvious when looking at the Perl grammar. perl is in fact the one authoritative source on how these ambiguous, context-based examples should execute. This fact might make you think that Perl and perl are tied more closely together than I have made out, and in reality this is very true. But at the end of the day perl is just a program that implements algorithms, like any other program! Even if these algorithms are not documented well, you could still - given enough time and intelligence - write them down in some kind of pseudocode (or even convert them to another programming language.) The logic in this pseudocode in combination with other formal grammar specifications would describe a Perl on paper. This Perl would then be implementable by any number of perls.

[ #106 ] Software Rewrites Permalink

Rant Added a year and a bit ago and last edited a year and a bit ago

I want to rewrite almost everything I use. I suspect lots of programmers do.

I know it's a delusion. Software is hard. There's no way I could rewrite common tools to be better than they are. Firstly I'm not a good enough programmer. Secondly, even if I were good enough it would take many decades to do them all. In my eagerness to solve the problems that frustrate me I would omit basic functionality that everyone else relies on. When it was brought to my attention I'm sure I would find that part so boring to implement that I would probably quit. Rewriting is both the only way to save many software packages and an impossible dream.

How do our tools get into such a frustrating state?

The Peter Principle is a special case of a ubiquitous observation: anything that works will be used in progressively more challenging applications until it fails. This is "The Generalized Peter Principle."

The "failure" I notice in tools isn't necessarily application-crashing stuff - it's something less measurable: inflexibility... unmaintainability... inconsistency

Inconsistent commands or options tick me off. I want to categorise them and rename them logically. Git and Vim - I'm looking at you. When git branch -a lists all branches but git tag -l lists all tags I get annoyed. When using Vim and zC closes all folds recursively, zf creates a fold, zO opens the folds but zG is a spell checking command... I get annoyed. These differences might seem minor to you (and they are used as small examples off the top of my head) but they reflect an underlying problem with the structure of features in software. As soon as little differences like this exist then it means you have to spend time learning each command by itself rather than being able to generalise your knowledge about one command over to another using the same syntax.

That being said, I love the power of both those tools. But when I start thinking about changing the command syntax it snowballs into changing the operation of those commands and eventually into adding/removing major sections such that it becomes a romantic idea to rewrite the whole thing.

Couldn't I just get involved in the open source project and submit my changes there? I could if I didn't want to rip out old functionality and totally rename things. Projects tend to be against that sort of thing because that would break thousands of shell scripts that rely on them. Introducing new features is fine - GNU is always doing it to improve the old Unix standards (awk -> gawk etc) - but it's very rare to see anyone breaking backwards compatibility. It makes sense not to. So here we are, back with the notion of the rewrite... the kind of rewrite where you call your new tool something completely different and push it into the cloud hoping for its adoption.

Rewrites are a very romantic idea. They appeal to the programmers mind because:

Being in charge of the direction of the code gives us a feeling of power
We love removing inefficiencies and generalising one concept across to others
We get an ego boost from being a project's rockstar

Not Invented Here is another name for the ego motivation behind some people's rewrite-fantasy.

I don't have any answers for the problem of wanting to rewrite everything... you can ignore the nagging voices and just get your job done with the tools you have but your creative energy will drain! Continually denying yourself the rewrite fantasy will cause you to become a hollow shell of a coder... a yes-man who always simply accepts the rubbish software around you... your drive to be a good programmer will fade and you will find yourself working for a marketing company writing email software for Windows.

Just joking... but you get my point.

I guess you need to find a balance. Sometimes (most of the time) you need to use what you've got to GET THINGS DONE. Every now and then you need to indulge yourself though... and rewrite the shit out of something.

[ #79 ] "I Don't Have Anything To Hide" Permalink

Privacy, Rant Added a year and a bit ago and last edited a year and a bit ago

Have you ever heard this justification when people are defending the invasion of their privacy?

It is wrong.

Everyone needs to hang on dearly to their privacy. Next time someone tries to use this defense, just say "Is that so? Then you won't have a problem with these questions:" and then ask them the following:

Can I have a copy of your most recent credit card statement?
How much do you weigh?
Can I see your pay-cheque?
What clothing-size and bra-size is your wife?
Which school do your children go to? What bus route do they catch and at what time?
What is your wife's favourite position in the bedroom?
How many times have you had sex this week?
Can I come along to your next doctor's appointment?
Can I look through your medicine cabinet and your sock drawer?
Can I check your browser history?
Can I have a copy of your last tax return?

Privacy is more than just protecting your passwords and your credit card number. It is about protecting sensitive information about your health, your children, your income, your interests, your love-life, ... anything that could be used to threaten, harrass, endanger, form an unfavourable opinion, or draw conclusions about you, your life or your family.

[ #72 ] Nebulous Requirements Permalink

Rant, Requirements Added a year and a bit ago and last edited a year and a bit ago

You are a techie. You might be an engineer or a web programmer or whatever. You get given requirements by management. Sometimes these requirements are explicit and measurable. This means that you know exactly what to do and how to do it and you can tell, at the end of the job, whether the requirements were met.

"Make a web form that lets someone create an account on our server" - while the implementation details are left up to you, the end result is testable and can be signed off by management. "Could I create an account with this form? Yes I could, thank you very much."

Other times the requirements you get from management are nebulous, flaky or incomplete.

When you get nebulous requirements, how can you tell that you've met them?

A colleague of mine was recently tasked with selecting "similar" data from a database in a search from a web form. This is an example of a nebulous requirement. How similar? Is 5 similar to 10? Is "there" similar to "their"? Is "tomorrow" similar to "tomato"? Is 10000 similar to 20000? Who knows?

Yes, there are algorithms that exist for this type of requirement. Soundex is an algorithm that can be used on English strings to match similar sounding items. Using Soundex, "there" and "their" are equal (which means that they sound similar as spoken in the English language.)

If the requirement my colleague got was to return similar sounding strings in the English language then he could have suggested using Soundex which is a well known, well tested and vastly used algorithm. Management would have agreed, then the requirement could have been changed to be "implement Soundex in the search."

Acceptance testing could then easily prove that this requirement was either met or not met because it is explicit, the algorithm is well known and input and output data can be used to test the implementation.

However the "similar items" requirement also applied to numerical data. My colleague devised a mathematical algorithm himself with an arbitrary percentage allowance between the sought number and the returned numbers. He also had another idea with standard deviations... I'm not sure what he ended up going with. Here's the juicy bit: it doesn't matter. The point is that managment said "SIMILAR!" and he had to decide what "similar" meant.

When my colleague has completed his code, will management be happy? If they want to they can tell you that it wasn't what they were looking for and you need to tweak it - and when would the tweaking end?

When someone has to maintain his code will they understand what was being attempted? A lot of nice commenting could explain what was supposed to be happening, but won't prove that anyone's idea of "similar" is being implemented at all.

If I was in the position of my colleague I would have explained to management that the requirements were nebulous and needed clarification with HARD DATA. When you are given a requirement you should be able to prove (to the best of anyone's ability) that the requirement was or was not met!

YOU DO THIS WITH USER ACCEPTANCE TEST DATA.

If I came to you and said I wanted the data in my application to be more springy, you would be in for a great deal of trouble:

You don't know what springy is.
You won't be able to prove that what you've implement is indeed springy
Management can refute the effectiveness of your implementation until cows come home, faces are blue and you are both fired for incompetence

Consider this however: what if I came to you and told you that I want the data to be more springy and gave you a list of 100 examples of a conversion between NON-springy data and springy data? And what if I said that to test whether your implementation worked all I was going to do was to test the data I had given you in the system?

YOU CAN NOW CREATE AN IMPLEMENTATION THAT YOU CAN TEST AND GET SIGNED-OFF. BY DEFINITION, YOUR IMPLEMENTATION MEETS THE REQUIREMENTS AS LONG AS THAT TEST DATA PASSES.

Now - obviously the springy example doesn't hold up well to scrutiny because it is a nonsense word in the context of data - it should be replaced with whatever nebulous requirement you are given today :-)

If my colleague had done one of these two things:

Gotten the requirements changed so that they involved implementing specific algorithms -or-
Obtained a set of user acceptance test data that would be used to determine whether his homebrew algorithms worked

then, I believe, he would be in a better position - a position where he could prove that he had done his job.

Note: Whether management were or were not satisfied with my colleague's results is not the issue. I personally think he did a great job technically - but the issues I am concerned about are provability and maintainability of code.

[ #24 ] Gauging Developer Progress Permalink

Rant, Agile Added nearly two years ago

Here's a ridiculous article that attempts to espouse the benefits of counting lines of code (LOC) by comparing software development to house building: Counting lines of code can help measure progress

Development is often compared to building a house. I’ve built a house, and I can tell you that while I did want the bricklayers to be finished on time, that wasn't enough to satisfy me when I had to write the check. Generally, I’d check their progress every day, and I would notice how much brick had been laid.

What this moron and his ilk fail to understand is that software engineering is a different style of work to ... well, anything. Let's take his building example. A house, 99% of the time, is a solved problem. A builder who has worked on previous houses already knows exactly how to build the next house (unless it's a fancy architectural-type house, or it's built on sand or made entirely from glass etc.) In software development however, if a large problem has already been solved before then that module (or open source library) can be reused. Therefore new work being done by devs often (not always) fits into one of these categories:

Linking found libraries into existing code
Refactoring
Bug-fixing
Glue code
Customisation (settings etc)

... and this work can be equally or more important than new features written from scratch. It can also be less important - it all depends on the value of the features, not on the lines of code it takes to write them.

Both refactoring and bug-fixing can sometimes actually reduce the number of LOC in an application, because existing code is studied for any inefficiencies or errors and newer, simpler, more modular, more elegant code replaces it. If you were measuring LOC based on changed lines then this would be taken into account, but if you are measuring LOC based on new lines then this work would be ignored. Let's say we were indeed taking into account changed lines, and we have an example where 5 lines were altered and 3 lines removed to give an efficiency increase of 300%. Should these 8 line-changes be given equal weight to 100 lines of code elsewhere that only adds one small feature? You would have to compare the value of the small feature to the value of the efficiency increase!

Compare ticket values, not lines of code written.

Sometimes a bug can be so insidious that it can take a whole day to find and fix - and it might be a one-line fix. Should a developer be punished for this, just because his daily LOC measurement is low? What if the bug-fix was a show-stopper? Even our old friend William Gates thinks that would be a bad idea:

Measuring programming progress by lines of code is like measuring aircraft building progress by weight.

**Scrum** takes care of the problem by examining relative complexity values during sprint planning meetings. Once a list of tickets (items to be worked on) have complexity values you can estimate their development time and equally distribute the tickets among developers. Progress assessment is now easy, managers just need to check the burn-down charts... and that eliminates the need for misguided and archaic measurement techniques like LOC checks.

[ #22 ] Python Pet Peeves Permalink

Python, Rant Added nearly two years ago and last edited nearly two years ago

The current Python (I do not know about Py3000) has some annoying features (annoying for me, not for you - you love them.)

How's this for counter-intuitive:

>>> a = ['f', 'a', 'i', 'l']
>>> a.join("")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute 'join'
>>> "".join(a)
'fail'

... the join method, to me, seems like it's in the wrong area. If I have an array (list), I want to join it together with a string. I don't want to get a string, then join it to an array.

More counter-intuitiveness:

>>> import re
>>> pat = re.compile('^\w\d+$')
>>> s = 'b45343'
>>> s.match(pat)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'str' object has no attribute 'match'
>>> pat.match(s)
<_sre.SRE_Match object at 0xb7c6e598>

... when programming and I have a string, I want to see if the string matches a pattern. I don't want to get a pattern and see if a string matches it. My suggestion could involve regex stuff being built into the string library - or not, depending on implementation. Either way is fine.

How about pypi, the place where Python packages live. Say I want a lib to do SSH, and don't know what the package is called. I ask pypi and I get a list including:

conch
paramiko
pushy
etc...

Contrast this to searching CPAN which is the place where Perl's libraries live. I get this:

Net::SSH::*
Bundle::SSH
etc...

CPAN is clearer.

Here's another thing: you can't do assignments in conditional tests. Observe:

>>> if (a = (1 + 1)): print "yay"
  File "<stdin>", line 1
    if (a = (1 + 1)): print "yay"
          ^
SyntaxError: invalid syntax

... compare to Perl:

$ perl -e 'if ($a = (1 + 1)) { print "yay\n" }'
yay

[ #13 ] Idiot Proofing is not Good Permalink

Rant Added nearly two years ago and last edited nearly two years ago

Wikipedia says:

In modern English usage, the term "idiot proof" describes building, organizing or writing in such a way as to be usable to someone of lower to average intelligence. Defensive design. The term came into use in the late 1970s to early 1980s.

Look at this Idiot Proof Website. It is a brilliant satire of this dangerous and backwards concept.

Programmers and designers are often told to make their products idiot-proof for the final customer. I contend that when you make something idiot proof you encourage idiots and punish intelligence. Users should be rewarded for their intelligence by having flexible interfaces that grow with the knowledge of the user.

The problem with idiot-proofing is knowing when to stop. How much do you dumb an idea/interface/feature down before it appeals to your local idiot, and how much does this restrict and frustrate an informed or intelligent user? The process is purely speculative and therefore inaccurate. It offends intelligent users who are willing to read manuals, brief introductions or even contextual help to further their expertise.

This issue is tied in closely with backwards-compatibility. Cars don't have reins to be backwards-compatible with horses. If you continually pander to idiots and backwards-compatibility then you are forever stuck with the same problems and constraints of current and past technology - you are preventing progress.

Does this mean that I think users should need to read an entire manual before starting to use software? No. It simply means that users should not be restricted by a designer's hazy guess at how intelligent they should be. Striving for clarity and eliminating needless complexity is not idiot-proofing. Clarity and power should be the end game for software or design - not restriction, constriction or condescension.

Shaw's Principle: Build a system that even a fool can use, and only a fool will want to use it.

(Nothing Older) ... (Nothing Newer)

Colophon

Django Python 960.gs Git Vim NetBSD Nginx

The Author

This is the blog of Brad Willis, a software engineer living in Brisbane.

Links

ChoppingBoard, DaveMisc, Project365, RageQuit