Creating Music playlist and playing in Roon

Here is a 30 second video showing me clicking on the "MakeMusic" script icon (built upon my universal playlist generating program mplay) which generates a playlist called Music (it takes about a second, the script window stays open for 5 seconds so you can see it worked) and then playing this playlist in Roon.

mplay Generation 2: playlist generating programs

I've been working on automatic playlist generating programs now since January 2021.

I originally had just one program, mplay (make playlist).

Now there are more programs: splay (shuffle multiple playlists together), tplay (tell about what actually played, and truncate playlist or play history), and shufflelinks (shuffle multiple "playlist" folders-of-links together).

These programs can be compiled in scripts, making playlist generation either more intelligent (such as removing the items that didn't actually play from the playlist history before creating a new playlist) and/or combining playlists (or folders of links that serve as playlist for programs that don't handle playlist) together in different ways.  (Playlists can also be simply concatenated together using the system command cp or equivalent so I didn't write a program for that.) 

Also mplay itself has advanced considerably since the 2022 software release on Sourceforge.  I hope to release this new version before the end of 2023, possibly fixing some of the current "gaps" between what it does and what it really should do.

One key new feature is the ability to specify ALL of the files in the specified folders.  As in "just make a playlist with all of the files in these folders."  You don't have to guess or predetermine the total number of files that would be included.  Since the playlist history is unchanged by this operation, it is neither read nor updated (however, it might be better if it reads to playlist history first, to put unplayed items at the head of the playlist, currently it just ignores if itms have already been played or not).

You can also apply an file age criterion in order to make a playlist consisting of only files newer than a specified number of days.  That can be combined with the ALL option to easily create a playlist of ALL the new files.  A script can concatenate this premier playlist at the beginning and/or end of a main playlist.  (Another useful feature not yet implemented would be to make a playlist of ALL the as yet unplayed files.)

But I've found it useful also to mix the new files into a playlist of older files.  There are two basic approaches to this kind of mixing:

1) Dense Shuffling selects one (or some specified number) of items from each playlist, which are then ordered randomly (or not) into a new segment of the output playlist, with additional segments added until all the items from every playlist are included.  If any playlist is exhausted early, it is reset and reshuffled.  This works very much like mplay itself (though currently only for non-audio files, since audio files are currently treated as if they were all in one big folder, with no other option).

2) Sparse Shuffling mixes a smaller playlist into a larger playlist so that each item in the smaller playlist is included exactly once (or some specified number of times) in the end result, using random placement.

Currently splay does only Dense Shuffling, and shufflelinks does only Sparse Shuffling.  It would be good to have both available in both programs, and/or to combine the programs to work on any combination of playlists and links-folders, to generate either a new playlist or links-folder.

tplay is very useful in the case where I've been playing an audio playlist in Roon, but switch to playing other items.  If I go back to the playlist in Roon, it does not remember the last item played but simply starts all over from the beginning.  Using tplay, I can remove the items that have already been played from the playlist.  Or, I can remove the items that were not played from the playlist history and create a brand new playlist.

Thinking about HDCD again

Using DVD-5000 to decode HDCD on played on PD-75 as digital transport

 I've been testing a pair of the DVD-5000's I originally bought for use as living room DACs.  They are both for sale on ebay as I write this.  I am reminded of the fact they are very good, and according to objectivist standards should not be audibly different from the very best DACs.  They also have the ultimate R2R chip made by Burr Brown, the PCM 1704, which was discontinued in 2012.  By objectivist standards, it should be no different than a good Sigma Delta chip.  But it is different in objective ways, for what that's worth.  I love the DVD-9000 for HDCD decoding but don't use it for anything else.  I would have been tempted to use DVD-9000 for midrange DAC until I discovered that I needed 3 DACs of exactly the same design for my tri-amplified system.  Even if I adjusted the delay times for one particular sampling rate, when I changed sampling rates those adjustments would no longer be valid, because the latency of nearly every device varies with sampling rate.

Faced with that problem, and the (at the time) skyrocketing price of DVD-9000's, I opted to get 3 DVD-5000's instead.  Those would be basically the same thing, I predicted.

But for probably no good reason, I just never fell in love with the DVD-5000's the way I did for the DVD-9000.  And it was much more convenient to have multiple DACs with level controls, and smaller.  So I ended up with 3 Emotiva Stealth DC-1's, which are quite fine also (technically they measure much better than the DVD-5000's, but once again that difference should not be audible).

I still use DVD-9000 for decoding HDCD's (because my Oppo BDP-205 doesn't do that).  So I tried using the DVD-5000 (which doesn't actually play discs, but connected as a DAC to the digital output of a Pioneer PD-75, optically) and I found it does indeed do very well with HDCD's, just like my DVD-9000.

But it is different.  When playing an HDCD the output level never exceeds the output level from an ordinary CD.  The DVD-9000 actually plays the level expanded portions of an HDCD up to 6dB louder than an ordinary CD.  In contrast, the DVD-5000 does what every other HDCD player I've ever tested (other than the DVD-9000), it lowers the average level, just so so the most expanded peaks of HDCD reach the maximum CD level, but no more than that.

When my brother-in-law George first heard the HDCD on a DVD-9000, he was (unusually I'd say, because he never matches levels by measuring) shocked and appalled that that HDCD boosted levels 6dB above normal CD levels.  He felt that was unfair.  HDCD was all just a cheat, he insisted.

Well, as I've started to explain in many other previous essays, it's much more complicated than George and almost everyone thinks, because of inter sample overs.

(And of course, George was also wrong that HDCD players boost the level beyond that of CD's.  Though for the longest time I wondered if earlier HDCD players did do that boosting, and it changed when players stopped using the PMI chips.  I had it reversed.  The DVD-5000 uses PMI chips and doesn't do boosting beyond CD levels.  The DVD-9000 uses a software implementation--like all later players--and does boos the level.  So it does not necessarily have anything to do with using the chip, it was just a particular approach the designers of the DVD-9000 took, which possibly was available to the users of the chip as well, but I haven't seen one that does.)

The ISO's on regular digital recordings can match those of even peak level expanded HDCDs.  And what's more, HDCD's don't seem to have such as big ISOs.  So it comes out about as a wash, the HDCD is just giving a kind of engineered peak, and regular CD's are giving us extrapolated peaks.  That dynamic range in the analog output was there and needed anyway, HDCD's (when boosted) are just using that dynamic range for real music dynamics, rather than extrapolations from what may be just high frequency ringing.

Now I also know, that SACDs may in fact be the worst offender of all.  They can have the highest peak output above the nominal 2V level.

But I'm wondering if it was the boosted HDCD level (compared to other players) that led me to fall in love with the DVD-9000 in the first place.

(I think HDCD done as intended (and with the filter control*) would have been a great idea...as an open standard, and would have enabled a way to bypass the loudness wars, using the HDCD encoding to provide the uncompressed version which for which the compressed version would be heard without it.  Now we're simply stuck with HDCD as necessary for reproduction of a significant number of fabulous sounding recordings, which are better heard when available in high resolution.)

(* The filter control somehow seems less necessary as HDCD's don't seem to have the transients that provoke excessive ISO's.  Somehow they achieved that effect without a post-filter change.  It will be necessary to do more investigation.)

What's going on with Poulenc in DSD

I've always loved the Linn recording of the Polulenc Concerto for Organ.  There is no question, it's very dynamic (as a whole album, much less so in individual tracks).  The SACD layer has a marvelous sound.  I have not listened to the PCM layer in awhile.

It has by far the highest peaks above the nominal 0dB RMS level of any disc I have encountered (+7.5dB).

I have been accounting for these peaks as Inter Sample Overs.  That might not be the correct description of what is happening on SACD's.

On PCM recording, the highest peaks seem to be associated with the leading edges that produce the most pre- or post- ringing.  This makes sense, as they basically represent very high level high frequency content which 'propels' an interpolation of the signal to go way beyond the normal boundaries.  Only high frequency content can do that.  Even when reaching 0dB, low frequences just reach the top slowly, over very many samples, and don't change enough from one sample to the next to cause an ISO.  (That was why I measured 0dB with 880 Hz, a low enough frequency not to produce ISOs.)

But on DSD...there is no pre- and post- ringing.  That is of course the beauty of it.  You get smooth looking curves that visibly look like the original wave forms.  (The eye is being deceived.  In actuality, those smooth looking curves are obscuring vast high frequency noise, which is often making the curves look smoother than the real thing.)

Here the entire Poulenc album on the DSD layer, recorded at 96kHz.  Notice that the very highest peaks which give the album very high dynamic range only occur in the last track.  Just a couple other tracks have higher peaks at all.

Poulenc (entire album)

Now lets take a look at what may be the highest peak (there are actually a bunch of them as you zoom in) in the last track:

Mostly, it doesn't look that extraordinary.  There's no sharp edge, no pre- or post- ringing.  However, there is an interesting notch on he bottom channel at the very peak (near the cursor).  Given the scale, that represents very high frequency information.  It may be a telltale sign that some sort of limiting or other thing was done just here.  But the notch itself is tiny in comparison with the overall peak, which doesn't look out of place in its surroundings.

Dynamic Range in Recordings is not a well defined concept

Dynamic Range in audio equipment seems relatively easy to understand.  It is the difference between the peak level and the noise.  

OR, it's the diffrence between the peak level and the lowest resolvable signal, which can sometimes be 20dB or so BELOW the noise level.  (Signals can be resolved from noise using spectrum and/or gestalt analysis, and the human auditory process does both.)  This is a little less well defined.  How capable is the spectrum analysis?  And the gestalt analysis is not very well defined at all.  Gestalt analysis is how we can identify discriminate different sound sources by other aspects of their quality than frequency, such as by their rhythm or randomness or apparent location.

 Either way, dynamic range in audio equipment is now specified as being done in the presence of signal, thereby requiring the signal to be filtered out from the product before analysis.  This is not hard to do now that we have ready access to things like FFT.

Sometimes this kind of Dynamic Range is called Signal to Noise Ratio in the context where we are looking at the noise of a particular component instead of an entire system.  And also perhaps when were are not measuring it in the presence of signal, which was the traditional way of measuring it (and what I usually do on my test bench).

But where it really gets thorny is when we are talking about the Dynamic Range in audio recordings.  It bugs the heck out of me when people just don't get how poorly defined this concept is (even though we now have 'Standards' and tools for measuring it) and how much, therefore, how much it depends on assumptions, heuristics, algorithms, psychoacoustic research, and the like.

For example, you could rightly claim that nearly every DDD recording has at least 96dB dynamic range, if you defined dynamic range as the difference between the peak level (which is almost always near the maximum level, defined as 0dB, on digital recordings) and the lowest level in the recording (such as in a fade out near the end of a recording).  

Also, since sound is wave-like, and audio is based on alternating currents, there will be zero crossings in every small section of recorded audio.  So it's always going down to zero somewhere and actually quite frequently!

Even if you removed silence-before-the-first-downbeat, silence or mechanical fade outs, and zero crossings, you may often find tiny near-silence gaps even within in highly compressed program material.  Even ruling out the two considerations above (fade outs and zero crossings) we can just look at the low level those relatively small gaps (for example 250ms, enough for 250 cycles at 1000 Hz and about where we hear gaps as gaps and not just clicks) and say "see, there is plenty of dynamic range between these 250ms intervals and the peak levels."

What the whole family of standards, methodologies, and tools for measuring "Dynamic Range" in recordings is really about is determining how much compression has ruined particular recordings, or alternatively how much compression can be used without making things sound bad.

It's a matter of heuristics.  They do this by comparing some quantity of peak levels to some quantity of less-than-peak levels.

But the quantities involved are not something that it intuitively obvious, but rather related to what is found to be 'interesting' in that it correlates to things sounding over-compressed.

So it is that some Dynamic Range methodologies use histograms and others use percentiles, things that are not intuitively obvious to self appointed audio gurus.

For example, the famous DR scale is based on a comparison between the peak level (fine, that's intuitive enough) and the average of all levels that are in the 20th percentile of highest levels or above (rather counter-intuitive).  IOW, it's comparing loud vs loudest, whereas you might think that dynamic range should compare loudest and softest (but then we are back to the 96dB of digital recording systems, etc).  But the presumption seems to be that "almost of the recording is at or below this level" and it should have at least one much higher peak to be "dynamic."  It seems to me there are other ways of being dynamic.

Why not the 50th percentile, or the 80th percentile???  Apparently because choosing the 20th percentile makes the tool identify recordings that sound overcompressed as compared to the genre of recordings they come from (which score differently even if no compression were used).

Furthermore, the DR tool analysis uses short non-overlapping segments of 3 seconds each.  You could get significantly different results if you defined those segments as shorter, longer, or overlapping.  You could argue for the merit of 250 msec intervals, which could produce a radically different result.  Likewise for 10 second intervals.

You might also think that something like Equal Loudness spectral curves (such as the famous Fletcher-Munson) ought to be applied...and indeed some tools use that...but the DR tool does not.

I was inspired to research and write about this after reading a poster (Tank) at Hoffman's discussion site talk about the difference in "dynamic range" between the 30th and 40th anniversary reissues of classic King Crimson albums.  I very very much like the 40th anniversary series, released on high resolution DVD-Audio, but this poster claimed the dynamic range on many albums in the 30th anniversary series was higher (and he like them better).  (I have not listened to the 30th anniversary series, and it's not "high resolution" so I might not bother.)

It really bugged me that this simpleton poster pushed back against a recording engineer (Plan 9) who was arguing with him, claiming that the dynamic range issues were easy to understand and the engineer was just full of BS.  The poster showed tiny graphs of different songs and said "See!"  But anyone who has done any amount of audio editing knows that things can look very different at different levels of zooming in.  

Sadly the engineer didn't do a great job spelling out his case, and after being denounced by Tank he just shut up.  But he did mention micro-dynamics and macro-dynamics and as far as I can tell, these are real things and not just BS.  Right now I couldn't define either one and I still need to read and understand more in this area.

[I previously wrote similar but less complete comments about the R128 used by Roon just a few posts back.  R128 does make slightly more sense to me than DR, perhaps just because I misunderstood DR.  It still seems to me highly arbitrary, though Roon's system works pretty well in practice for keeping a constant "level," which I noted then is not immediately obvious to me how that is done from the R128 dynamic range rating which is about range and not level.]

Inter Sample Overs vary with digital process ?

[Correction: Subsequent data has called most of the theoretical speculation below into question.  It turns out that SACD's do not have consistently low ISOs, they can possibly have the highest ISOs of all, as I detected with the the 'Poulenc' SACD released by Linn.  This is an orchestra with a large pipe organ, and if I had associated low ISOs with higher quality, let alone previously believing all SACD's to have low ISO, I would have believed this to require a minimal 2.0dB headroom for ISOs.  But in fact it set my current record...it required a a mind boggling 7.5dB headroom.  And it was somewhat inconsistent.  It might work with as little as 6dB headroom....or it might not.  It apparently produces a peak around the same size as the 96kHz sampling interval, and how high it registers depends on how much of the peak occurs in any one sampling interval.  So there is a large random element to how much headroom is required.  I've also noticed that on some DVD-Audio discs with 96kHz sampling rate.

It does still seem that Reference Recordings and 'Discipline' (the brand behind the 40th Anniversary King Crimson DVD-Audios) produce consistently low ISOs.  I would venture they seem to know what they are doing.  But the peak ISO level seems not to depend on DVD-Audio vs SACD.  Nor does it even have a fixed relation to music genre (rock vs classical) or does it necessarily have anything to do with the recording "Dynamic Range" or how much compression was obviously used---though it's possible some compression was slipped into the Poulenc at key moments and that's what gave it such high ISOs.]

I have begun fairly systematically copying all my SACD's and DVD-Audios into 24/96 copies for my hard drive.  And I'm noticing a few weird things.

Remember that I set my gain structure so that with an 880 Hz tone recorded at precisely 0dB (generated by Audacity) a +4 level would be the highest possible before clipping.

But for actual recordings, I have to set the level between -2.5dB and +2.5dB, giving them at least 1.5dB extra headroom and as much as 6.5dB appears to be needed in some cases.  This headroom is required for inter sample overs (ISOs) where in between the samples the signal peaks higher than 0dB.

Benchmark claimed that ISOs could be as high as 3.01dB and that many digital decoders failed to allow sufficient headroom.

BUT I am finding 3dB headroom is way insufficient.

EXCEPT, in some cases it isn't.  Some discs seem to cluster around 2dB headroom required, up to 2.5dB in some cases.  Which discs are these?

1) Reference Recordings HRx (174kHz/24bit)  seem to only require 2-2.5dB headroom.  These are some of the best sounding recordings ever.

2) SACD's generally only seem to require 2dB headroom  (I would have expected SACD to be the 'wildest,' but in actuality, it's the 'tamest.')

3) The DVD-Audios in the King Crimson 40th anniversary DVD-Audio boxes, which seem only to require 2.5dB headroom.  These are truly spectacular in audio quality (or in the case of Court of the Crimson King, merely way better than ever before).

On the other side?

1) Many hot sounding recordings on DVD-Audio, including Elton John, and especially Steely Dan.  (I've long noticed that.  I figured Steely Dan cranked up the compression so high it's bleeding out the ISOs). 

I wrote this off for awhile, but now I'm getting a pretty clear feeling that the recordings which require the least ISO headroom are the ones that sound the most natural, laid back, and 3D.  The recordings that require the most headroom sound highly processed.

You may be astonished to see some of the pictures when I post them.  Having the huge ISO's means the rest must be scaled back in a recording, losing dynamic range in the midrange.

I think the excessive ISO's occur when the the anti-alias filtering is insufficient, and full scale high frequency garbage gets into the digital encoding.  THAT's what's causing so much overshoot.

DSD has such a high rate it captures and controls all the HF crap by design.  So SACDs do not suffer from excessive ISO's.

Likewise, the PMI analog to digital encoders used by Reference Recordings, which must have superb filtering.

And whatever King Crimson was using in the 40th anniversary DVD-Audio set.

Now, it would be interesting to know what Steely Dan used in such things as the Everything Must Go DVD-Audio, that produces such high ISOs.

Perhaps it's not the digital process, but the amount of processing (including compression) used before digital encoding that is the culprit.  But the biggest ISOs look too big for just that, IMO.  Still, the sonic variation might just be coincidental, the least processed recording just happening to use the digital processes which produce the least ISOs.

This is also a function of the Oppo BDP-205, which is apparently not headroom constrained itself.  But I've found the height of the ISOs not to change with different reconstruction filters.  I think they are inherent in the data and and the degree of oversampling used, with higher amounts of oversampling 'revealing' the true ISO height (because you are filling in more of the points in between the points).

******

Update:

Now it appears that that the ISOs in DVD-Audio discs are all over the map.  They can be < 2.0dB, for example, in Queen's A Night At The Opera.  They can be < 2.5dB such as in every King Crimson 40th Anniversary DVD-Audio box.  Or, they can be as high as 6.5dB, as in Steely Dan's Everything Must Go, Elton John's Goodbye Yellow Brick Road, or Frank Zappa's QuAUDIOPHILIc.

SACD's still appear to be consistent at around 2.0dB, and everything from Reference Recordings is there too.

The Audio Hobby

The audio hobby means different things to different audiophiles, and isn't that wonderful?

Not always, perhaps.   It can be made awful in many ways.  People can fall victims to unfounded beliefs that cause them to waste time and/or money.  It can lead to the virtual inability to listen to music anymore, in extreme forms of audiophilia nervosa.

It can lead to endless bullying.  I think generally the best attitude is non-judgemental: live and let live.  However all the same I believe audio is filled with frauds of many kinds, both on the big and small levels.  You probably already know I side mostly with the audio objectivists as to what kinds of audio beliefs are founded and which are not.  All the same, I don't take it as my mission to change anyone's mind.  I suppose, in some cases, even I could be wrong too.

Anyway, nowadays I think generally it's a waste of time to compare good amplifiers, DACs, cables, power conditioners, or anything of that ilk.

All have been tested endlessly by audio objectivists getting only null results.  With my lack of patience, I'm unlikely to do better, if I follow all the proper procedures.  If I don't, then the result may be meaningless anyway.

My normal result with good amplifiers is this:  I start the auditory level matching process by matching the apparent loudness with both amplifiers to make it identical in fast A/B testing.  Once I have matched the level, I attempt to match the quality of the sound.  If one seems to have more bass, or highs, I assume that's because it's actually louder.  In this second phase I make only the smallest adjustments, 0.25dB at at time (I'm fortunate to have a first generation Emotiva Stealth DC-1 with 0.25dB adjustment.  That means, on average, the best case is within 0.125 dB of being exactly correct, which is close enough to pass the 0.1dB minimum in my experience.)  Ultimately, on every amplifier I've tested, I can make them sound identical simply by matching the level that closely.

Now I did not do such procedures when I thought for several years my then go-to amplifier, the Aragon 8008 BB, was sounding a bit harsh.  Back in those days, I found myself avoiding listening to the FM for very long.  It drove me insane.

I later found the distortion had risen to 0.7% because of low bias.  After bias adjustment, I got it back to 0.07% and sounding fine.

But I know that from years of experience and not A/B tests.  And measurements which make that experience believable.

And that's another thing.  One should often believe measurements, when they are meaningful and honest.  Not necessarily specifications.

Anyway, distortion can be a factor down to 0.1%, so products having higher than 0.1% should generally be avoided.  (In electronics, anyway, where it's easy to do better.  There's hardly any speakers that can do as good as that.)

In audiophile land, there are often electronic products with higher than 0.1% distortion.  And sometimes they sound better.  I think what's happening is that in some cases products with predominantly 2nd order distortion may fix recordings that were made with high amounts of 3rd order distortion.

Furthermore, boosting the amount of 2nd order distortion, which tends to occur with zero feedback designs (feedback works best at suppressing 2nd order distortion) can add additional "spaciousness" and air to recordings lacking those things because of poor production.

Things like this may work on some recordings and not others.  It may work best on recordings that are fairly simple, like a few instrumentalists.  Not on works of great complexity, like a full symphony orchestra.

Euphonic adjustments are like that.  Generally it's best to stick with low distortion, wide response, low noise, because that's the combination that works overall best on everything.  Basically what the objectivists say.

Other claimed magic requirements in design, however, are sold on the basis of faulty comparisons, typically failure to match levels very well.

Anyway, if feedback free amplifiers, and electrically charged cables, or whatever makes sense to you, go for it.

The best is when we're not bullying people over such things, one way or the other.

Even being forced to make a decision is a kind of bullying.

We are generally not designed to discriminate among audio reproduction systems.

We don't have a 'memory' that works very well for making such comparisons.  We don't store 'experience A' in anything like the raw form that would make for a good comparison with 'experience B.'

To be reliable at all requires, as the objectivists always say, instantaneous A/B switching.  Other than that, perhaps exhaustive training.

Furthermore, always being assigned to make the equipment comparison detracts from the process of having the most enjoyable and enlightening experience from the music, appreciating the music itself rather than arcana of possible sonic differences caused by different audio equipment.

Fine, the uber subjectivists say, just see which piece of gear gives you that most transcendent audio experience.

That's basically impossible, because each time you listen to the same piece of music you get a very different experience.  Symphony orchestras often like to prove this by playing a Premier (first ever) performance of some work twice, sometimes even without warning.  Few guess it was an identical repeat.  The identical music doesn't provoke an identical response.  And for a very important reason.

The you listening to any work the second time is now older and wiser, having already heard the music before.  The brain has already stored memories and made new connections.   Usually that opens up entirely new realms of experiences.  While closing down others.

(Curiously a work with more 'movements' can be heard more times without seeming repetitious.)

As many have often opined, that is what audio should be mostly about, the music, and quite often isn't.

(However in line with non-judgmentalism, I prefer to say that any mode of enjoying an obsession with audio reproduction is fine.  If your thing is building amplifiers that test conventional theories -- fine.  If your thing is arguing about whether such things can or do make any difference -- fine.  As long as I'm not to much detained by your obsessions.)

But along those line, I fear the excessive denigration of standard audio engineering practices, as often occurs in the writing of audio subjectivists (including the one I always loved to read anyway, Harry Pearson) tends to promote rather than discourage audiophilia nervosa and therefore lack of being able to enjoy music.

But some people swim in one thing or the other, so whatever works for you.  Doing 'comparisons' is also a way of just hearing things twice, which may itself be beneficial.

I have dubbed such demonstrations magic shows, and typically enjoy them even when (that is, all the time) my core beliefs are unshaken.  

I myself only drifted into near audio objectivism in my late 20's, after doing carefully constructed experiments I felt would 'confirm' my beliefs in tweakdom.  That's the path of many noted audio objectivists.  First they were fully taken in, then they decided to do some tests.

After the recording, DAC, and amplifier problems are 'solved,' is there anything left?  Two things, the loudspeaker/room (or headphone) interface, and the selection of the music itself.  Neither is a problem that will ever be 'solved.'