How Sheet Music can change: Going from the printed page to a tablet computer like the iPad

For a long time I’ve wanted to be liberated from “sheet music” on paper. I want a computer device that can show me only exactly what I need at the moment for ensemble sight-reading, rehearsal, and performance, without all the drawbacks of printed paper. There are myriad potential advantages; I’ll bring up some of them here. I want to concentrate on how a musician actually sight-reads and what they need to see, and removing the distractions of what they don’t need to see.

I want all my music notation in a device like an iPad which I can mount on a music stand or hold in my hand in the case of singing in a choir. So somebody needs to develop apps for tablet computers that facilitate this: there are many under development.

What we don’t need at all is something that displays PDF files, because ultimately PDF files are formatted like printed pages, with a height and width, with margins and page breaks. You may not have thought this through, but those concepts would be anathema to reading music notation on an electronic display device, and I’ll explain why.

Figure 1. The traditional page layout
Figure 1. The traditional page layout

 

Formatting a music notation document for printing on a page, such as US Letter or A4, is not going to cut it for viewing on a tablet device. We need some sort of way to easily reformat an existing score with page dimensions into something that a person can easily read on a tablet device, with a foot-controlled mechanism for advancing to the display of the next bars, ideally some sort of wireless Bluetooth foot pedal. Note that I did not use the term “page turn”; I’ll get to that later.

Kindles and iPads are really popular, but I read tons of content I purchase from the Amazon Kindle store on my 2nd generation iPod touch with its 480 x 320 resolution screen which is only about 3 inches by 2 inches (7.6 cm x 5 cm) in size. I’m very comfortable with this for the simple reason that I understand that in reading text and nothing else (with no multi-tasking going on) it is a fact that the human eye and brain don’t look at a whole page at a time. The reality is something quite different: the human eye and brain only look at groups of about three words in a single glance. So if you have a mechanism for scrolling the text easily, there’s absolutely no need to have a display of a whole page if you are only dealing with text and no illustrations or diagrams.

The appeal of the Kindle is for people who don’t realize how the eye and the brain work in the first place and have a mistaken notion that they actually need to see a whole printed page on a screen large enough to display the whole page in one go. They have this mistaken notion because they’ve been holding books in their hands all their lives and they’ve never really thought through the process of how their eye and brain work. Obviously the reason that the page in the book is the size that it is, is because of the needs of printing paper, cutting it and binding it, and making something that fits well in the hand. An electronic reader device removes all these physical restrictions.

It’s my belief that as the public becomes accustomed to the new paradigm of reading a book on an electronic device, the screens on newer models of these devices will get smaller, until the Amazon Kindle of 2021 will be the size that the iPhone is now.

Note that I’m talking about a device strictly dedicated to the purpose of reading a book that is all text with no illustrations or charts or diagrams. I’m not talking about a device that works like a desktop computer in which you want to have separate windows open and visible simultaneously, such as one window for your email, one window for your calendar, one window for your Web browser, one window for your word processor, one window for your real-time stock market ticker feed. For that, I have a desktop rig with two large wide-screen monitors that total 3360 x 1080 pixels, so I can do many things at once and have them all visible. But for reading a book that is purely text, with no distractions, my older-generation iPod touch at 480 x 320 is not only adequate, it’s ideal for me.

Now reading music in rehearsal and performance is different. In some cases you are only reading one staff, your own part, but in some cases you are reading an open score, where you need to see many staves in a whole system in parallel across the vertical dimension, if you are a conductor or choral accompanist. And as a chorister, I actually want to see the whole SATB-plus-piano-reduction-system at once, because I rely on seeing and hearing the other folks’ parts so I can find the pitch for my next entrance. Orchestral instrumentalists are accustomed to seeing a “part extraction”, only one staff and only one part, and seeing a notation symbol like a 31-bar gathered rest, and sitting through those 31 empty bars with no visual indication of what is going on in the parts playing all around them. But choristers can’t abide by that kind of notation–I know I can’t.

I hope that tablet device music notation reading applications free us from the whole concept of the “sheet” or “page” altogether. If you only need to read your own one-staff or two-stave musical line, (maybe with chord symbols on top) you should be able to toggle off all the other stuff on the page and just scroll or flip along on your part alone. There should be no need to see more than one system at a time on the display screen (no multiple systems per page) and again you should be able to turn that off, so you can zoom in on just the staff line or arbitrary system grouping of staves that you actually want to see at that instant.

Let’s look at the cognitive aspect of reading sheet music–what actually happens with your eye and brain.

When people are taught in school how to sight-read sheet music, they are encouraged to practice reading one or two measures ahead of the measure they are actually playing or singing. In other words, you learn to develop and use a mental “real-time input buffer” so that at the instant you are playing one note or chord, your eye is actually focused on the note or chord one or two measures ahead of the one you are playing. As the seconds roll by, while you play note after note, your eyes scroll across the system, and you view and anticipate what is to be played in the ensuing seconds, so on and so forth, moving forward in real time. Ideally you develop the ability to turn the page two bars before the last note in the last bar on the last system of the page, so you can see what notes you will be required to play in the first bar at the top of the next page. It’s a bad idea to wait until you play the last note on the page, and then try to snap the page turn instantly and land right on the first note of the next page. If you do that, you will probably flub the next phrase you need to play.

Hopefully you figure out every opportunity to look up from the score and look directly at the conductor, such as when you play a long held note or chord, or when you realize that your part has a four-measure rest before your next entrance. This is particularly crucial in genres like choral music where tempos change or speed up or slow down dramatically all the time, and the director of course determines this in real time, and may in fact improvise his tempo changes with each performance.

The point I am building to is this: A tablet device music notation reading system should abolish the concepts of pages, page dimensions, the system break and the page break altogether. A score played in such an app should always be in “Panorama” view and never in “Page” view, to use the metaphor used in the Sibelius music notation program.

Figure 1. The traditional page layout
Figure 1. A traditional PDF page layout, two systems on the page

 

A score should scroll side-to-side, with the next one or two bars ahead of the note being played being visible at all times, with never a “jump” across a “system break” or “page break” at all. And a capability like the “Focus on Staves” feature in Sibelius should always be available in any playback mode as well.

 

Figure 2. Panorama view in Sibelius.
Figure 2. Panorama view in Sibelius. Music scrolls left-to-right. There are no system breaks.
Figure 3. Using "Focus on Staves" to only see the Tenor line and piano reduction
Figure 3. Using "Focus on Staves" to only see the Tenor line and piano reduction

If the music notation computer document being loaded into the tablet application has formatting for pages of certain dimensions and aspect ratios, with system breaks and page breaks, the tablet application should ignore those and reformat the music on the fly. Alternately, a music notation creation program like Finale or Sibelius on the PC or Mac should have an option to output a second version of the formatted score with all the pagination and breaks removed, specifically for exporting to a tablet reading application.

I wish to remind you of a famous quote from Henry Ford, the creator of the Ford automobile. He was asked if he took what his customers desired into account in designing his products. He replied, “If I had asked my customers what they want, they would have told me ‘a faster horse.’ ” In other words, when he first started Ford Motor Company to make the Model-T, his potential customers had never seen or driven an automobile, so they couldn’t conceive of what a car was or why they could possibly want one. If asked to describe what they would like to buy in a mode of transportation, they would only think in terms of horses and carriages, not of automobiles.

Steve Jobs, and later Jonathan Ive, used exactly the same philosophy in developing the first Macintosh and the first iPod and iPhone. Apple explicitly does not use focus groups when coming up with new product categories or concepts. Instead they get visionary designers who can come up with something so unprecedented that consumers could not possibly even understand what the thing is or why they would want to use it until they actually hold the thing in their hands or sit down in front of it, at which point they say, “Aha! This is insanely great and I can’t live without it, even though I had no idea beforehand that I needed such a thing.” Well, nice work if you can get it, but that’s how product revolutions are born.

My point is that if you develop and deploy a device with a totally new paradigm in hardware and software to consumers, they might just catch on. On the other hand, you can take the conservative route and do like Amazon with the first-generation Kindle (with credit for prior art to Sony), and bet on the idea that consumers just won’t be able to deal with something at all unless it looks more or less exactly like a page in a printed paperback book, with more or less the same page layout, size, and aspect ratio. So far that’s working for Amazon.

But I contend that this page/size physical metaphor will not work for a music notation reader. Paper scores work fine; the only problem is the amount of stuff you have to carry around, and the page-turning problem. In order to convince a significant number of consumers to replace paper scores with an expensive electronic device, you have to provide something dramatically more user-friendly than paper scores. This means two things: removing the need for the player’s eyes to jump around over a large page, and removing the page-turn problem.

The player would benefit from the ability to focus their eyes on a fixed point or scan across a relatively small area horizontally only, never vertically, while the music scrolls horizontally, thus the reference to “Panorama view”.

I’ll leave you with two things to think about. First, how to handle the repeated section, first and second endings, da capo, dal segno, and coda?

 

Figure 4. Confusing notation we can live without
Figure 4. Confusing notation we can live without

These are all artifacts from the ink, paper and printing era, as they exist only to reduce the amount of work that the copyist/engraver needs to do to prepare the score, and to reduce the amount of paper needed to print the score. They have no benefit to the performer. They mightily increase the amount of thinking that the sight-reading performer has to contend with, increase the need for the eyes to flit hither and thither across the score, and increase the dreaded page-turn problem. They foster confusion and mistakes in rehearsal and performance. So my idea is that if you take a music notation document and open it up in a music reading app for a tablet device, that app should never display any repeats, da capo, dal segno, or coda notation. The application should simply expand the whole score as one continuous panorama with nary a need to go back any number of measures and re-play a section or visibly skip around anywhere. This especially goes for multiple verses of lyrics.

 

Figure 5. Multiple verse lyrics require repeat signs
Figure 5. Multiple verse lyrics require repeat signs

There should no longer be any need for “verse 1, verse 2, verse 3” displayed one under the other, under the same system of bars, with a repeat sign at the end of the verse.

 

Figure 6. Use Panorama side-scrolling and never use repeats
Figure 6. Instead, use Panorama side-scrolling and never use repeats

Second, how to handle the scrolling “real-time input buffer” situation? Can you set up a continuous scrolling marquee display at a certain speed, so the performer never has to tap on something to “turn a page”? I’m skeptical about this, because of the issue of tempo change, accelerando, deccelerando, ritardando, rallentando and the like from the director. It might be possible to have a foot switch array with multiple buttons to speed up or slow down scrolling, but that would be even more work for the performer, and if the scrolling display gets out of sync with the director, the performer will be lost. I suppose that a very elaborate system could be devised whereby every single performer in the ensemble has the same brand of tablet device and music notation display application, and the conductor controls and modulates the scrolling speed while all the performers’ tablet devices scroll the music notation in sync with the conductor’s tablet device. But that’s probably creating a technological train-wreck scenario if it breaks down or slips out of sync in a performance. So we may well need a display that shows a fixed number of bars that don’t move, until a performer taps a button or steps on a switch that flips ahead to the next static small group of bars.

I’d really like to get a discussion going on this post, so by all means, comment.

Be Sociable, Share!

0 thoughts on “How Sheet Music can change: Going from the printed page to a tablet computer like the iPad”

  1. Hi,

    My partner and I own a private music lecture and thus at the moment creating a web-site to aid my kids here http://readmusic.org

    Do you mind when I quote a few of your articles as long as My partner and I include credit and also sources back to your web-site? Our blog is in the specific same area of interest as yours and our students is likely to truly benefits by a lot of the information you provide in this article. Please let me know if this fine with you.

    Thank you!

  2. Hi Wheat,

    Saw your post on James Humber’s blog and wanted to see what you were talking about on your blog. If your talk at NAMM has a PowerPoint (or Keynote), would you be willing to post it later?

    I completely agree with your post, but I think the paradigm shift needs to be gradual as we move away from paper to LCD/e-ink.

    First, we need to get tablets into the hands of everyone, preferably thinner. I know instrumentalists that would like a much larger version.

    Second, the choral octavo, in general, fits nicely on the iPad. There are some situations where a PDF shows larger than the original score. As a transition device, or in situations where publishers seem to have no interest in reprinting (or re-transribing) “old” publications, this might be a need. Several apps on the iPad, such as forScore and UnrealBook work very well in this regard. There are some publishers who don’t print in octavo format (choral), such as Earthsongs. This can cause issues with the clarity of scores on a PDF music reader.

    Third, we’ve got to address how copyright is handled and how composers are paid. My fear is that every publisher (take Hal Leonard, for example), will create their own multi-platform app, and then the performer will have to try to jump between apps in a concert, for each publisher’s proprietary format. I’d like to see a situation where either digital “sets” could be purchased, or something like the Harry Fox agreement where publishers/composers would get paid every time a score was used. One thing is for certain…an average price of $1.95 per choral octavo isn’t going to be tolerable if everyone has tablets and there is no printing, shipping, binding, or paper involved. I’d like to see the price lowered, the composer receive more, and the publishers be able to keep the same level of profit as they currently receive (without the aforementioned costs). I saw that Florida is expecting all curriculum (I think they mean “core” subjects [as much as I hate that terminology]) to be digital by 2015. Shouldn’t music be on the same timetable (I’ll be blogging about this later).

    Fourth, there’s still the Finale/Sibelius divide. Sibelius 7 fixed much of this with Music XML export in addition to import, but Finale has no iPad version of Scorch, which sounds so tempting that if my family had not just bought a new car, I’d drop the $199 to invest in an competitive upgrade today. Scorch might be a valid answer for some users…but how to get all that printed music into Scorch? For a high school music teacher, who might be working with 40 to 50 pieces at a time, that’s a lot of data entry with Sibelius or Finale, even if scanning is far improved. And is that legal, even if you own paper copies?

    Fifth, there are some neat MusicXML resources, like SeeScore, that display MusicXML files (with text), and allow you to turn off staves as desired. You can also adjust the size of text by pinching or zooming (very iOS-ish).

    And finally, as I review iOS apps, one of the key items I feel are necessary for any music reader…PDF or otherwise…is the ability to annotate music. The PDF music readers have this…I’m not sure about Scorch (SeeScore doesn’t have it yet). I don’t think that any piece of music is valuable to the conductor or the musician if they cannot annotate as necessary through the rehearsal process.

    1. Yes, when I wrote this blog post, Avid Scorch was not out yet. Now that we’ve seen Avid Scorch, everybody knows that annotating scores is the big thing that is lacking. I’m sure the good folks at Avid are hard at work on this, but cut them some slack–they just released Sibeilus 7, and they have to deal with Mac OS X 10.7 Lion as well. These are exciting times.

  3. I’m getting ready to write a grant for choir tablets. Your blog helped open my eyes to the issues. I would like to add the ability to broadcast a music quiz or some kind of notation test, have them take, score it immediately and have their grades end up in my electronic grade book. For my junior high choir this would be amazing.

  4. I realize this post is dead, but I came up with an idea for a solution regarding the last part, and I want to make it public for others like me who discover this page.

    There are many modern devices and electronic instruments that utilize a “tap tempo” function, where the user literally taps a button (or – often in the case of effect pedals – a pedal) to the beat of the music just for a measure or two; the device uses this input to sync with the tempo of the music being performed.

    Utilizing this concept, a simple one-button footswitch – ideally with the depth and clickiness of a computer mouse button – could interface with the reader and control the scrolling speed. And perhaps holding the button down could signal the scrolling to pause.

    Seeing as many – if not most – instrumentalists already have a habit of tapping their feet to keep tempo, this would be convenient for the musician and entirely solve the flipping/scrolling issue.

    Anyway, this (side-scrolling music notation tablet) idea is innovative and a sensible improvement from printed music. I not only hope, but expect to see this technology replacing paper sheet music in the future.

  5. A few years ago some group was on KPFK, Los Angeles, heralding the “end of all written music” ..boy was that a heated argument with me..I had to point out the vast library of great movie music that influenced them…but they dismiss this.
    I have conducted smaller groups always with notation.
    What if the iPad craps out, mid concert? ..or what if it even slows down as often my iMac does and needs rebooting?

Leave a Reply

Your email address will not be published. Required fields are marked *