Month: April 2013

Here’s a blog post by a talented math teacher, about a time in his life when he was bad at math, and how hard that was, and always is, and why.

This blog post has resonated in a raw, painful sort of way for me. So, for my third daily writing session, I’m going to write about it.

Please understand that this is a painful topic for me; what I’ve written about it could easily be described as whiny, or emo. I have lived a life of relative privilege, and I’ve gotten to do some things that many of my friends would give their eye teeth to do. What I’ve written could easily sound like I’m not aware of that privilege, and of course, sometimes that’s the case.

Also, fair warning: this post is long.

I’m writing it, though, to convey what my life felt like, to me as I was living it. I mean to communicate that even someone who is good at something can still feel wracked by insecurity about it, can feel bad at it.

So, in one way or another, I’ve been struggling with math anxiety for most of my life.

I was a smart twelve year old kid who didn’t know I had any talents, lying to my mom that I’d read the algebra textbook in my useless homeschooling time. I just read novels. I was sure I was an idiot at math.

I was a thirteen year old who had literally guessed my way through the state math test, sitting in a community college classroom waiting for the first day of “college algebra” (remedial upper high school stuff), pretty sure I was going to fail it. It took the best teacher I ever had to erase that fear, and he erased it in a matter of days.

I was fifteen, with an ego, having aced a bunch of calculus and diff-eq classes, but I was struggling with a Number Theory book given to me by a UT professor. I was worried that because my parents wouldn’t let me go to Princeton, I was already the end of my math career. I was failing to understand the book on the first try, and having overdramatic “maybe this would all be solved if i was dead” type thoughts, as fifteen year olds are wont to do. I gave up on the book and became afraid of the subject.

I was sixteen, dropping my first class at UT, that same Number Theory. I dropped it because I was afraid of it, because I hadn’t understood the book on the first try. I remember feeling such shame in the counselor’s office, as if she had any reason in the world to judge me for it. I remember being worried that the department was going to kick me out of school for being a fraud, worried my parents were going to give up and pull me back home.

I was seventeen, and I had aced number theory and four or five other undergrad math classes, and I was taking my first grad class, Complex Analysis. Nobody told me there was an undergrad version I was supposed to take first. I tried, I tried hard, and then I gave up. I failed like the blog author failed: catatonically, silently, for months. It was a bad time for me. I stopped talking so much to my friends and family. I lost all my confidence. In some ways I never recovered from this one. All the mathy or academic things I’ve ever done since then give me a faint hint of anxiety, a faint hint of possible failure that I try very hard to ignore.

I was eighteen, having aced another ten math classes (they were all I took), including two more grad classes, and preparing to apply to grad schools, but struggling to do any work, pushing myself to do any work, because I still wasn’t over the C in the first grad class. I worked hard, did eight hour homework sets on saturdays, but did it in a stressed out way, shouting at my family and friends on a regular basis, panicked that letting go would mean giving up. I was afraid of getting another C. Grownup me can’t even imagine being afraid of that anymore.

I was nineteen, with wind back in my sails, even more grad classes aced, a research paper in pre-publication, newly arrived at Cambridge, starting to do work on an eighty year old unsolved problem (Artin’s Little Conjecture), running up against it as hard as I could, still feeling so anxious about my talents, investing so much of myself in it. I told myself, regularly, that solving this problem was the litmus test of whether I could do great things in my field, whether I deserved to be there at all.

I remember asking the favorite prof about it in his office, and getting a careful “you’re going to risk your doctorate” (he was right) and a dismissive “and anyway you’re just a Part III student, I don’t have time for you” (he was a jerk)… and that wrecked me. It’s pathetic to admit, right in keeping with the blog post, but I don’t think I’ve had a day of true mathematics ambition since that day.

This is the kind of thing that happens when you care very much about something and you are deathly afraid to fail it. Little things are greatly magnified, and you handle them poorly and derail yourself.

I’m not even done.

I was twenty, putting off preparing for my Part III exams, entirely, until the last three or four weeks, because I was sure I was going to fail. I was headed back to UT, having not gotten into MIT’s grad math program, having chosen Texas from homesickness, and I was already sure I was a failure as a mathematician, just for the little reasons described above. I passed my tests and got my degree, but I would quit mathematics altogether within a year.

I was twenty one, a math grad student at UT, but completely unwilling to work. I didn’t sleep well, I didn’t leave the house often enough. I never ever showed up in my shared office. I stopped talking to my friends in the department (sorry, Todd, Shel, etc. It was a hard time in my life). In one of my class finals, I became so convinced I couldn’t do the work, so unwilling to even try, that I angrily drew a comic (“Signs you shouldn’t be a mathematician”), turned it in, and stormed out.

The department chair told me, in a kind and gentle way, that I was risking probation, and asked if I was okay. I had nothing to say; a month later I went back and asked him if I could quit, said I wanted to transfer to being an undeclared undergrad. He was kind about this. I felt so much shame. So much completely unnecessary shame.

I’d like to make an aside to point out how little any of this had to matter in my life. None of this needed to be important. None of matters as much as the first time I kissed my wife, the first time I juggled fire, the best day I’ve spent with my brother. None of this will matter on that future day where I first see my child. But we choose what matters, and we have to choose it again every day. Sometimes we choose wrong.

I’m still not even done.

I was twenty two, in the middle of a second undergrad in either chemistry or computer science (I tried two things, because I was worried I wouldn’t even be able to do at least one of them), having dropped numerous classes in both, struggling to get out of bed and do my work, struggling to feel like I could do any useful work at all. Feeling old, feeling like a public failure, like the blog post says. Feeling old and failed, at twenty two.

I was twenty four, twisting and drifting in an internship at Amazon, feeling too old to be an intern anyway, in a team where the manager had left after three weeks and we were unmanaged. I was struggling with real professional code, convinced I couldn’t make sense of it, that I was finally going to be run out of comp sci for not being an engineer.

It was a twelve week internship and I got stuck on some small stumbling blocks… and I sat doing nothing for the middle five weeks of that internship. I was just plain afraid of the anxiety that came with being stuck. It took a coworker taking me for a walk around the block and pointing out that I was in trouble of not getting anything done, and asking where I was specifically stuck and pushing to help me, took all of that, to get me to finish. I did finish with some good stuff.

Twenty five was a good year for confidence 🙂

I was twenty six, at Microsoft in the wrong group, feeling (or imagining) the daily contempt of the engineers around me for not being able to operate in the Hotmail environment, struggling to make myself show up at all. At night I was listening to my friends Dan and Avi talk about Stanford grad life, looking through my mental window like an urchin outside a banquet, wanting that experience back but feeling like I had closed it off to myself forever, because at this point I couldn’t even remember most of what I had worked on, and because… how could I go back and fail that again? I couldn’t even follow a lot of Avi’s conversation without a lot of refresher and explanation and stumbling.

And then I committed to a plan to turn this around. I would go to Seattle, do a different kind of work, use the University of Washington to get back into academic life. For a while it was great! I had the promise of mathematical work (eventually) at Amazon, even turned down that same promise at Swype, and was taking a grad quantum chemistry class, something totally outside my expertise, but also totally outside my old social groups and history. I had no fear, so I did well, got the highest or maybe second highest scores in the class.

Then I failed. I couldn’t handle Amazon and a grad class at the same time. I wrecked up my health by giving up exercise and eating all kinds of bad food, got heavier again, started getting exhausted at random times, as one does when one gains a lot of weight quickly, and finally I just had a really bad morning for thinking, and I completely failed my grad chemistry final. Like, bottom fourth of the class, after being the top.

The professor still really liked me, and I could have recovered for sure, taken the hit and continued the plan… but I felt that old failure anxiety, that old “I am really, secretly bad at this” anxiety. I bailed on my plan completely and utterly. Some of my friends ask whether I am considering going back to grad school… and I talk about my maturity level and whether I’ve aged out of math, but behind that talk I’m just frankly really damned uneasy about trying again and failing again. This math/academic/science anxiety hasn’t really ever left me.

I was twenty eight, finally starting the aforementioned mathy part of my Amazon job… sitting sullenly in my corner playing Tiny Wings, or looking out the window. Unwilling, more or less completely unwilling to believe in myself again after such a bitter failure.

I was thirty, this afternoon, at Google with an awesome machine learning project, now five or six months overdue for a variety of life and personal and team and company and whatever reasons… with good, clear and reasonable steps to take to move the project forward… unwilling to do so because I think maybe I will be fired, and because I don’t actually know machine learning well enough, and because I am still afraid of trying and failing.

These are some of the moments where I held myself to the most unreasonable standards, afraid that because I wasn’t the prodigy I set out to be, I wasn’t going to amount to anything. I still struggle not to feel that, right up to today. I’m still comparing to an imaginary ideal.

But why? The world is an amazing place, whether I am here to enjoy it or not, built on millions of beautiful incremental advances, the kind I could aspire to. There are always going to be people with talents totally beyond mine, whether I admit it or not, and ultimately it’s my choice how to feel about all this, so I can’t really justify feeling so bad.

I’m going to try not to feel so bad.

This is some very scary footage from the Texas Fertilizer plant explosion:

The man in the video has been roundly and soundly criticized for having his child in the car, a few hundred feet away from the plant. People can’t imagine someone could be so stupid.

I have a different take. In fact, the video makes me feel guilty for all the other times I’ve held people (including myself) to impossibly high standards.

What he did is dumb, but not impossibly dumb. I could easily see myself doing that. Before the explosion, it’s just a fire. A chemical plant fire, admittedly, but it’s something you could forget.

See, I think we all get mad about stuff like this, get mad at each other, get mad at ourselves, because our  internal model of a human being is something like this:

Of course, we don’t think we’re all that special, nor do we think we’ve reached our potential… but we do think we have a consistent dignity, a certain nobility, some intelligence.

We do have those things, much of the time. But far more often than we’d like to admit, this is who we really are:

Every day, maybe almost every hour, we do something this profoundly stupid. Each of us. In the last hour I’ve forgotten that I was actually writing this piece. In the last hour, you’ve might have forgotten you were having a conversation. You might have put your keys on the little table behind the couch where you won’t see them. Maybe bumped into a table on your way to the kitchen. Both of us probably spent a moment re-affirming our belief in something as implausible as running through walls.

I think the man in the explosion video was having such a moment. I think almost everybody I’ve ever judged was having such a moment, a derpy moment, a hyucksome moment.

All this, in a one minute theme song.

This is you, this is me, this is everybody, unpredictably, many times, every day, for our whole lives. With, you know, some Luke Skywalker thrown in.

Pity, not judgement. Pity, and maybe good natured laughter. That’s what I keep telling myself.

Here’s another game, clone of an old Popcap favorite of mine, the little known Alchemy.

Alchemy is a stone placing game where you match rune stones by color and symbol to make gold.

Here’s the game, and as always, if you care, here’s the source.

Oh, and here’s Popcap’s unpleasant Java based port of the game.

My game has four levels of increasing difficulty. I’ve already spent a lot of time happily play testing it; the last level is wicked hard, at least for me.

Let me know what you think!

from fnmatch import fnmatch
import os
import sys

if len(sys.argv) < 2:
  print "Please supply a path argument; this is the folder path to your music folder"
  exit(1)

path = sys.argv[1]

total_mp3s = 0
total_folders = 0
total_weird_folders = 0

for (sub_path, folders, files) in os.walk(path):
  mp3files = [x for x in files if fnmatch(x, "*.mp3")]

  total_mp3s += len(mp3files)
  total_folders += 1

  if len(folders) > 0 and len(mp3files) > 0:
    print sub_path
    total_weird_folders += 1

fraction_weird_folders = 0
if total_folders > 0:
  fraction_weird_folders = total_weird_folders / float(total_folders)

print "Total number of mp3s: %s" % total_mp3s
print "Number of folders with mp3s and subfolders: %s" % total_weird_folders
print "Total number of folders: %s" % total_folders
print "Fraction of folders with mp3s and subfolders: %f" % fraction_weird_folders

Let’s go through this line by line:

from fnmatch import fnmatch
import os
import sys

Here, we’re asking python to import a bunch of extra tools for us to use. ‘os’ is operating system. This gives us powers to do operating system type stuff, like listing folders, moving files, stuff like that. ‘sys’ is system. This gives us the power to read things from the command line. ‘fnmatch’ is file name match. It will let us do filename wildcards, in this case ‘*.mp3’.

We’re going to run this script from the command prompt. What is the command prompt? It’s the old-style part of your computer where you can enter commands by keyboard and stuff happens. How to use it for: Mac Windows

So, at a command prompt, we’re going to type something like

python countMp3s.py /Users/mcarlin/Music

to run our program. The first part tells the computer to run python. The next two parts are called arguments; they’re extra pieces of information for python to use. The first argument is the name of our script, countMp3s.py. With this, we’re telling python to run our script. The second argument is going to tell our program where to find music.

if len(sys.argv) < 2:
  print "Please supply a path argument; this is the folder path to your music folder"
  exit(1)

This code checks to see whether the script has been given two arguments. If it hasn’t, the script asks the user to try again, and quits.

Sys.argv is the list of arguments. It should have two things: the script name, and the location of music.

path = sys.argv[1]

Sys.argv[1] is how we get the location of the music. The bracketed 1 means “get me thing 1 from Sys.argv”. If we had said Sys.argv[0], we would have gotten the script name. We put the location of music in a variable named path.

total_mp3s = 0
total_folders = 0
total_weird_folders = 0

Here we’re creating counters for the number of mp3s, number of folders, and number of weird folders. They all start at zero.

for (sub_path, folders, files) in os.walk(path):

os.walk takes a path (the location of our music) and “walks” through every sub folder. In each case, you get three things: the path to the sub folder (which we call sub_path), a list of folders inside this folder, and a list of files in this folder. Since we’re in a for loop, the next few lines of code (everything which is indented) will happen repeatedly, once for every sub folder.

  mp3files = [x for x in files if fnmatch(x, "*.mp3")]

This one might be self explanatory. It’s equivalent to saying: please give me every file in the list of files, if the file matches “*.mp3”.

  total_mp3s += len(mp3files)
  total_folders += 1

len(mp3files) is the length of the list of mp3files. However long that list is, that’s how many mp3s are in this folder. We add this to the total mp3 count. We also add one to the total folder count.

  if len(folders) > 0 and len(mp3files) > 0:
    print sub_path
    total_weird_folders += 1

If this folder has more than zero sub folders and more than zero mp3s, it’s a weird folder (remember, that’s something I’m looking to count). We print out the location of this folder, and add one to the weird folders count. Okay, we’re done with the loop. The rest of this stuff happens just once, after the loop is over and everything has been counted.

fraction_weird_folders = 0
if total_folders > 0:
  fraction_weird_folders = total_weird_folders / float(total_folders)

We know the number of weird folders, but I also want to know the fraction. So, assuming the total number of folders isn’t zero, I divide number of weird folders by total number of folders to get the fraction.

print "Total number of mp3s: %d" % total_mp3s
print "Number of folders with mp3s and subfolders: %d" % total_weird_folders
print "Total number of folders: %d" % total_folders
print "Fraction of folders with mp3s and subfolders: %f" % fraction_weird_folders

Now we just print out all the numbers. You can put a variable into the print statement by use of %s (for strings), %d (for integers), or %f (for fractional or floating point numbers).

That’s it! That’s the whole program. I ran this program on my music collection, and here are the numbers I got:

Total number of mp3s: 25191
Number of folders with mp3s and subfolders: 113
Total number of folders: 13226
Fraction of folders with mp3s and subfolders: 0.008544

So I have 25,191 mp3s, and just under 1% of my folders are weird. Yay!

You can do a lot of really powerful things with the ‘os’ tools in python. You could remove all the duplicates from your music collection. You could remove “The” from the beginning of every file name. You could copy all the folders with more than 10 files to a different location.

If you have an idea for something to do with these powers, leave me a comment. I might write another few of these tutorials based on reader input.