Spam trends

This is going to be boring, unless you like reading about the rate of SPAM going down. SPAM, as you probably know, is a huge problem. Not only does SPAM usually carry some kind of nefarious offer of discount goods and services, but many times it also includes malware- software to be installed with a malicious intent.

SPAM, it has been estimated, is one of the biggest wastes of time in the corporate world. One report concluded that while on average a normal employee may waste 30 seconds of time per incident of SPAM, an executive may waste up to 30 minutes per SPAM! This is because most executives feel they need to get others involved, or because they try to unsubscribe. Everybody else chalks it up to an artifact of using the Internet, and moves on.

The Washington Post and several other news sources reported yesterday (Nov. 12, 2008) that McColo Corp.- an American company used as a front to send SPAM- was taken offline. Almost instantly, every SPAM scanning service around the world noticed a dramatic drop in SPAM volume. A graph collected by SpamCop.net can be found here. If you view this graph the week of writing, you will notice that the date of Nov. 13, 2008 shows a chart of up to 30 SPAM per second. This is a marked difference from Nov. 7, 2008, which reports over 70 SPAM per second reported.

IronPort, which is one of the world's most used SPAM filtering servers, also collects statistics from each of their devices purchased. These units are spread around the world, so they tend to get some very interesting data! IronPort has released a statement noting that SPAM levels almost instantly dropped to around 1/3rd of their previous rates!

All of this got me wondering what my SPAM graphs would look like. If the true majority of email on the Internet was from this company, I should also notice an immediate drop in quantity. That was my hypothesis.

The graph above shows my company's email volume by number of emails (blue), number of SPAM (pink), and number of virii (red). Finally, the quantity of bytes is shown behind the other columns (green).

As you can see, our company email tends to follow a moderately regular trend. Weekends are particularly low-volume days for business email, as we are closed. But notice how close the Mail and Spam bars are? Weekdays bring the quantity of legitimate email up, but the SPAM rises as well. Either way, theraw numbers show the for the past month, my company has received 268,984 emails from the outside world. Of this number, 207,887 were SPAM. That's an average of around 77.3%! This equates to 4908 Mbytes of data, or roughly a DVD worth of pure, worthless, garbage.

But what about the decrease? Looking at the right-most columns, you will no doubt notice the difference between the SPAM and Mail bars. What are the numbers? Well, for Nov. 12, 2008 I received 2,748 emails at our gateway. These were emails that passed the initial checks I do, which do not appear in the graphs or counts. Of this 2,748 emails, 1,313 were classified as SPAM. That's 47.8% for an entire day's volume! This is incredible! Today is just as good so far, at 46.1% SPAM.

Some other interesting numbers from my system:

Highest quantity Mail:
Oct. 7, 2008 - 6,026

Highest quantity SPAM:
Sept. 15, 2008 - 4,864 (81.9%)

Lowest quantity SPAM:
Nov. 12, 2008 - 1,313 (47.8%)

Highest percentage SPAM:
Sept. 20, 2008 - 93.4% (3,369)
Oct. 11, 2008 - 93.4% (3,560)

Lowest percentage SPAM:
Nov. 12, 2008 - 47.8% (1,313)

Learning Python

I've been learning a new programming language named Python for the past week. I've been serious about it since Tuesday afternoon when I bought a book on my Kindle about it. So far, I've written some test programs, and I've started work on some changes I'd love to see implemented in SBackup.

So, one thing that's very interesting about Python is that you can have an object (everything is an object) that implements methods. If you want to call methods within an object, but you don't know what they are, you can do something VERY creative! The following code returns something very nice:


import string

methodlist = [method for method in dir(string) if callable(getattr(string,method))]

So, what are we doing here? We first import a pre-built (and deprecated) package of code called "string". From there, we declare the variable "methodlist" is to contain the results from what happens to the right of the equal sign. That code is broken down as such:

Return the contents of the variable "method" for every time we can fill "method" from the results of the items within the code "string" IF the attributes of the code deem it to be executable.

Why is this so cool? Well, first of all, it saves you time in research! More importantly, though, it can drastically simplify the code you have to write in your applications. Let's say you write a program that imports a package of code you wrote that contains several executable methods. Each of these methods is used to return some results from data you feed it.

To do this is most other languages, you need to know what methods you want to run, and you must execute them one at a time. In Python, however, you now have a list of methods, which you can loop through as above, and feed them all whatever data you want! This means you can implement in one line what would take you 30 or more just for the above example! That's a huge payoff for something so simple, and smaller programs usually execute faster with smaller memory requirements.

Anyway, I'm enjoying Python so far, and I hope to be able to contribute some good code to the SBackup community and the Ubuntu community in the future.

Being skeptical

Since I was a very young child, my family used the parable of "Doubting Thomas" to describe my constant questioning and skepticism- never in a positive light. In the event you are not familiar with this particular biblical story, the Apostle Thomas doubted that Jesus was truly resurrected, and demanded to probe the wounds in his side before he would truly believe the other Apostle's claims. Upon seeing Jesus wit his own eyes, and the wounds in his side, Thomas was no longer a skeptic.

This particular parable goes on to detail how Jesus told Thomas that those who believe but have not actually witnessed are "blessed". I prefer to call them stupid. Being a skeptic does not mean I do not believe something I have not seen physically, but rather that I will not believe things that have no evidence to back their claim.

An example of my belief would be that the earth is, in fact, a sphere. I have not personally been in space to witness this particular fact, but I see evidence all around me. When a ship sails into the distance, it appears to rise, then sink below the horizon. This is a classic sign that we are on a sphere. Another fact that I believe is that astronauts have visited the moon. Again, I did not personally witness this event. In fact, I hadn't even been born yet! But the evidence to support the claim is so overwhelming that not believing would be ludicrous. It would probably also get Buzz Aldrin to punch you in the face.

Conversely, however, there are more than enough claims that I do not believe. Extra terrestrial beings visiting our planet is one of them. To the best of our knowledge so far, there is no way a living being could cross the universe's vast expanses in a lifetime. For that matter, crossing interstellar space would take several human generations! To cross the distance our radio signals have traveled in 100 years would take our current space vehicles 27,254 years (299,792,458m/s / 11,000m/s = 27253 years). Without some kind of physics altering technology, this is as near impossible as we care to conceive of. For perspective, this is about 10% of the entirety of human history.

One more claim I disbelieve without evidence is Homeopathy. This is the idea in alternative medicine that you can dilute a particular substance in water to non-existence, but the water will "remember" the original substance. When you ingest this water, your body uses like compounds to treat like ailments. As an example, you can take bee venom, dilute it in what is the equivalent of more water than the earth contains, and drink a vile this solution. Your body will now magically treat itself for bee sting allergies. If I know you personally, and you are allergic to bee stings, PLEASE do no try this. At the least, you will die from your sting.

The world is chock full of cheats, scoundrels, liars, and deception. To not be skeptical about claims made by others is to ignore the very thing that separates humans from the rest of the animal kingdom- complex reasoning skills. If I tell you that rainbow's are made by Leprechauns hiding pots of gold, but you learned in science class they they are made from light hitting water, which causes a prism effect, who will you believe? The simple answer is to do a simple test- use a water bottle to spray some water into the air in a fine mist. When you see your rainbow, look around quickly for leprechauns. If you don't see any, and you can reproduce this result time after time, chances are that leprechauns are a lie.

The moral of this story is that I am, in fact, a doubter. I am a true skeptic, and I have been since early childhood. I take that as a sign of a healthy, inquiring mind that is thirsty for knowledge. I also look at those that do not have this same inquiring mind to be ignorant, or at the very least unwilling to expend the energy it takes for simple fact checking. And in todays world, the stakes could not possibly be any higher.

Truth(s)

In a Socratic session during a high school literature class, I spoke about truth. There were misconceptions in the class, many of which came from the instructor- a new-age hipster, popular with all the students for his semi-counter-culture way of looking at life. He, as it turned out, believed there was no truth, but rather many truths. That is utter bullshit.

In life, there is only one truth. By very definition, there can only ever be one truth. Literally speaking, the truth is a statement regarding a fact. Facts are indisputable. Opinions, however, are open to dispute and debate.

This brings us to the misuse and abuse of the word truth. Many people use the word true or truth to represent a personal opinion on a particular matter. This, by the above linked definition, is wrong in every sense. Many people pose their opinion as the truth, and many people have opinions regarding facts. But, there is no opinion that can be called truth. There are only the two separate entities- truth and opinion.

As an example, I will use something we are all very familiar- literally the example from my Socratic session. Watter is wet. Now, you may have an opinion regarding how wet water is, or how watter feels against your skin as an individual. But none of those things displace the truth that watter is, in fact, wet. The very property that defines watter is that it is fluid. And any matter that is in a fluid state is considered to be wet, as a matter of fact. This, combined with the myriad other properties that water displays constitutes what we call wet. There is literally no debate to be had, because "water is wet" is a fact- a truth.

Take a moment now to consider the number of times someone has confused opinion with truth in your life. How many times has someone shared their opinion as though it was a fact? Try to keep a close ear on conversations throughout your day to discern who is reciting facts, and who is reciting opinion. Then, as a bonus, try to find who is reciting their own opinion, and who is regurgitating another person's opinion as though it were the truth. The results may very well shock you! But that's just my opinion.

Blame Canada...er...Government

By now, you should have seen the South Park movie. If not, go ahead and rent it. I'll wait. Done? Good. Like Rick James said, blame is a hell of a drug. Wait, no, that was coke. Either way, blame is just as addictive. Every day we try to blame others for our own mistakes and shortcomings. Why should the credit crisis be any different?

Recently, there have been a host of stories regarding the "credit crunch", "housing crisis", "banking bust", and several other financial issues that all seem to be contributing to a global recession. Now, I'm not economist, but I did graduate high school. In doing so, I had to take some basic economics courses, where I learned that spending more than your income is BAD. But apparently some schools don't teach this, and even more shocking is the fact that it's not common sense.

So, what's the problem with lending money to those that can't afford it? Well, for starters, any mortgage they qualify for is going to be completely bonkers with interest rates, late fees, and of course, quick foreclosure. You see, these people are what is known as "high risk" in the lending circles- there is little to no chance of them ever paying off a debt. So, one day a very smart lender decided to loan lots of these high risk borrowers a TON of money, and then sell those loans to other lenders. He made his money, so he is happy. The other lenders probably figured they were just victimized by a Ponzi scheme, and so they in turn sold the package of mortgages to some other lender. After a few years some smarter lender looked at the loans he owned and probably flipped his lid. Thus began the mortgage crisis.

On the other side of this coin, what's the harm in taking out loans you can never hope to pay back? Well, again, the interest rates will probably bury you well over your head in late notices alone. Moreover, you will eventually be spending every spare cent on your loan payments, and before you know it you will need extra money for some emergency- car repairs, hospital bills, home repairs, etc. These are all things you can count on happening with regularity, and never at a convenient time. So, there you are, a broken down car, no water heater, and then the bomb drops- You've been laid off. Now what do you do? You were barely scraping by before, you have no money saved for this situation, and finding a new job is getting harder by the minute. You, my friend, are going to be the recipient of a final notice to quit. And, as though that weren't bad enough, your credit rating is now through the floor.

I personally do not have a credit card. I find them to be tempting, and I know I will use them for some other purpose eventually. So, I avoid them at all costs. Apparently for me, one of those costs is the inability to purchase a home for myself. Why? Well, credit is a double-edged sword. With too much on credit, you are seen as a high risk. With too little on credit, you can't prove your ability to pay back lenders with regularity. So, it seems people that spend responsibly and are careful with what they do financially are punished for being wise. Then again, perhaps they aren't as wise as I thought?

Now we get to the good part. The blame. These lenders and high-risk borrowers have all blamed the government (pick a government, it's not just the US) for their problems. And why not blame the government? They should be regulating what you spend your money on, and they should regulate who can get loans, right? WRONG. It is your responsibility to spend, save, and manage your money. That rule applies weather you are a bank, a broker, or a consumer. That is how a capitalistic democracy works. Call me a conservative, but I don't need or want the government telling me what I should and should not spend on. Some level of consumer protection is fine, of course- after all, someone will always try to swindle another person. But, at the point where consumer protection becomes protecting the stupid...I'm sorry. They all need to learn a valuable lesson.

And what lesson are they to learn? All around the world government agencies are bailing out these lenders and borrowers. That's right, your tax dollars are buying homes, cars, and other valuables for people that spend money foolishly. Not only is the government bailing out the borrower, but the lenders that initially tried to dupe them into poor financial situations too! And we're not talking a bailout of a few million, either. No, no. This is into the BILLIONS. And that's just for one nation. All of the top-tier countries are doing this, which easily makes the total into the hundreds of billions, and even into the TRILLIONS!

For a bit of prospective, if I were to give you a billion dollars in $100 bills, it would take me 10 million $100 bills. If I could give you one bill per second, I'd be handing you cash for 115 days and just over 45 minutes. In other words, I could give you $360,000 per HOUR for nearly four months! I don't know about you, but my salary isn't even CLOSE to that figure. And that's just a measly BILLION. To get into the trillions I could give you the same amount of money for FOUR YEARS STRAIGHT!! Wrap your head around that for a second. There aren't even a trillion people on earth!