Plex Hack

There's been a widely discussed hack of the servers that Plex uses to host its blog and forums, but supposedly, not the servers that provide the connectivity to the Plex services. At least so far, my external access to Plex hasn't been affected.

When I started out on the journey through Plexland, one of the concerns I had was how dependent delivery of my content would be on a 3rd party. Of course it is! I've always had that lingering question about Plex and how, and whether, my access to my content would be affected by a disruption of their service. So even while I've not looked into their architecture, I understood that should the Plex server/service be disrupted, I'd likely lose my ability to access Plex outside of my own network. It's always a question of how much time/money/work you want to put into an effort like a media center, and there are other options, like Kodi, but Plex is amazing stuff, and for the price I paid for a "lifetime" (whatever that is these days) Plex Pass, is a great bargain if you choose to go this route!

For obvious reasons, Plex doesn't disclose its server/network architecture, but I found this well done blog post on how Plex implemented https, but also discusses some of the other bits behind the workings of Plex.

With the explosion of internet connections both personally and professionally, our devices and connections can probably never be considered secure, and especially so when you understand that when even a network security firm found it had been penetrated If they cannot defend their own (hopefully) highly secured network, what hope do individuals have (this is a rhetorical question)?

Yes, I did change my password. Yes, I'm watching for any 'extra' network activity on my Plex server.

P.S. 2015-07-06 - Plex forums are still down. No impact to me aside from having to re-log into Plex and the server, with every device. I noted that one person on Reddit mentioned disabling port forwarding. Since the hacker said he obtained IP addresses, I assume this could be those of the Plex user's server. Since I've changed my password, I don't see this as necessary and would really make me the victim of the hacker too, since I wouldn't be able to access my server content outside my home network.

Cellular Data Usage

March 2015 ended with my Samsung Galaxy Note 3 registering a bit over 76 gig in cellular data usage for the month. The previous month's usage was 56 gig.

90%+ of the usage was Plex streaming (in the car). This is the equivalent of streaming the data from 1 1/2 Blu-Ray movies to a phone, though most all of this was audio, and a bit less than 3 gig in tethering. I do have a 4G LTE phone, with a large screen. I use Wi-Fi where I can (Wi-Fi usage for the same time period was 46 gig.), but obviously when I'm truly mobile, well...

Curiously, my cell phone provider shows somewhat less usage (59 gig), so I'm going to investigate some apps that report data usage. I'll see if they are more consistent with my provider's reporting than the phone's usage reporting.

While it's not my intent to see just much data I can shove through an unlimited plan, it does illustrate how much some users do use and how quickly data usage is growing.  According to this study, the average user consumed 3.3 gig./month. I'll guarantee that if I didn't have an unlimited plan, I wouldn't be consuming a couple of gigs/day (vs. 3 gig. per month). Verizon's data for 2013 shows users consuming significantly less (less than 1 gig/month). TheFool.com, examining data usage from a completely different perspective (that of cell phone provider profit margins), indicates the average usage was about 1.8 gig/month-maybe more like what I'd expect.

Mobile Data Usage and Plex

One thing I've found since starting up my Plex project is that my monthly mobile data usage has skyrocketed. Now, I have a 4G 'phablet' type phone, so I probably chew up more than the typical smartphone user anyway.

One month, I used 46 gig of cellular data (not Wi-Fi), and probably 85% of that was from streaming Plex content.

This month, in the first two weeks of my billing cycle, it looks like I'm on track to beat that.<oops>. I've used almost 38 gig, and Plex accounts for 33 gig of that. I'm really just streaming music, so I'm surprised it's soaking up so much data.

Prior to Plex, I might have used 4-6 gig mobile data in a month, so streaming Plex is clearly driving the usage way up. I'll have to try Spotify and see if it's as hungry as Plex seems to be.

How much mobile data do you use per month? Mobile data usage is continuing to increase as we add bigger screened phones (and tablets with cellular data), with faster connections. It seems hard to find real information on average usage, but the rate of growth in usage seems destined for continued explosive growth.

Transitioning to Mac Mini for Plex

My decision to move from the Windows7 computer I was using to host my Plex server was driven by the instability of this particular computer (which has been an issue since I bought it, not of Plex). I wanted something that would be a smaller footprint, as the Win7 computer I was using was a laptop. Though it has a 1 tb drive, all of the media is stored on a Synology NAS, so the replacement didn't need to be anything with vast storage, but for purposes of transcoding (where necessary), did need to have a good processor and enough RAM.

I'd been waiting for Apple to announce their updating of the Mac Mini. I was really disappointed with their 2014 offering, as it seems they are really trying to show a clear differentiation between the Mini (as a low end product) and the iMacs. The new machines came only with an i5 and no upgradeable RAM. Apple RAM is expensive, so in the case of the new Minis, you have to order what you want when you buy, as the Mini has soldered RAM (no user upgrade). I decided to evaluate a reconditioned 2012 model, hoping there would be some availability with the launch of the new line.

I hadn't shopped for a reconditioned Mini, so I was surprised to learn when I went to look, there were none. Sigh. So, I set up a change detection search to look for me. I finally found some coming into the Apple store. I found a 2012 reconditioned i7 quad core Mini Server (2 x 1tb drives), with 4 gig RAM. I bought it (quickly, they seem to go fast), then later would evaluate the need to upgrade the RAM. I didn't really need the server version, as I already had the Synology. I may use the server app for some related purposes in the future, but right now, it primarily serves as my Plex server.

Apple setup is trivial, and I hoped that the Plex setup would be just as easy. Installing the Plex server itself was easy, and of course I had to change the port forwarding in the router to point to the new computer's IP address. Then I ran into my initial problem of mapping the drives on the Synology to the server on Plex and (since the Mini is so power efficient), finding a way not to have the Mini go to sleep (keeping my libraries accessible). In Windows, I had simply created mapped drives (M=Music, P=Photos, V=Video) and set up credentials for the Windows7 computer to be able to access the Synology. It always worked just fine once i got the credentials set up, but setting up drive mapping on the Mini and keeping it awake were something I hadn't thought about up until now.

I found people on the Plex Mac forum had similar concerns when it came to keeping the Mini awake. I settled on Caffeine. It seems to be working fine for me.

But not being all that familiar with users and mapping under OSX, mapping was a bigger unknown to me. I created users on the Mini for each of the drives and had them automatically login when the Mini started (using the Login items under System preferences). Once I figured that out,  everything seemed to work fine, though I have one intermittent problem, with one segment of my media library. Periodically, when playing something in my Music collection, Plex will seemingly start to play the song, but no music is played. This goes on for a few seconds, then it skips down to the next song, etc., but nothing is actually ever played.  Not particularly useful.

The first time this happened, I deleted the entry in Plex's library for Music (which was a mistake, as I'll explain). And re-added it. However, doing so means that Plex has to go out an reindex and match all of the media in the music library and add all of the album art. Oops. A reasonable number of my music tracks then had no artist or album art. Ok, manually go out and look for the album art and fix all of these.  Of course all of the music is then shown as "recently added", moving any of the other media types waaay down the list, since I had ~19,000 tracks in my collection at the time.

Then, it happened again.

I read through the forum and opened a post in the 'Ask a Ninja'  to look at the issue. Needed to upload a server log. This is way more help than I expected, since this is all volunteer work, helping others. The "Ninja" had a couple of suggestions, but I think the one I implemented that actually made the difference was to log back into the music folder on the Synology from the Mac Mini (on which the Plex server runs). Doing so suddenly fixes the issue with the music not playing. Don't even need to restart the Plex server! It's happened a couple of times since then, but since I haven't figured out why, the easy solution is just to log back into that remote folder (the Mini is connected to the network via ethernet, as is the Synology), and problem solved (for a while). The only other issue I've had is that even though the Plex port is forwarded to the (fixed IP) ethernet connection of the Mini, if WiFi is somehow turned off, Plex isn't accessible outside the house. This is still something to which I have not found a solution, and obviously I cannot forward the same port to two different IP addresses. So I just leave WiFi on.

I used the Activity Monitor to see how much RAM was being used with little more than the normal system processes and Plex, and was surprised to see it already using a swap file at this point, so I decided it was necessary to do a RAM upgrade. Prior to buying the Mini, I explored YouTube for a video on upgrading the Mini's memory (there are a couple of good videos). It seemed pretty simple. I bought the RAM from Crucial, upgrading from 4 gig to 16 gig (meaning that you have to replace the existing 4 gig RAM), after using their process (System Advisor or Scanner) to ensure I bought the correct RAM for my server. Cost was about $165.

The only problem I had was managing to get the bottom plate of the Mini to unscrew (twist off) so I could access the memory slots. It just didn't want to unscrew, but finally, did. There's nothing to upgrading the memory once you've done this, popping the existing RAM modules out and putting in the new. Fingers crossed, we'll hope there's no issue with rebooting...

All in, I spent about $1k on the Mac Mini and the memory upgrade. Add in the Synology (1513+) server and (baring any upgrades there), I've invested a total of about $3k in hardware.

No problems on reboot though, so now my little Mini is happily (and reliably) serving up my media. I only wish my Internet connection weren't so unreliable. But that's an issue I hope to address later this spring. But doing so, will mean reconfiguring the router. It's always something...

Venturing Into Plexland

2015-02-11
Venturing into Plexland

One of the motivations I had for buying the NAS was to centralize the media we had accumulated over a mountain of CDs, DVDs. They were never where you wanted to access them, if you could find them. While my strategy was fine, what I hadn't appreciated was the time and effort required to convert all of the physical media into files playable on a media server, and then keep populating the library with more content.

I had used XMBC (now KODI)  under Ubuntu and found it pretty capable, but it was limited in that I couldn't access my media outside my home. Through discovering the Plex app on my Android phone (a friend had shared their media library with me), I decided to try out the server  on my own. The app (either Apple or Android) isn't worth anything without media to watch, and you need a server for that. The server program itself is free, though if you want to watch on a mobile device (phone, tablet), there's an app you can buy to get the full features. You can also connect your Plex directly to your TV, stream it to a TV through a built in Plex app (my TV is too old for this), game console (PS3, XBox, etc.) or through a Chromecast (my solution).

Plex allows you to organize your media into some predefined categories--TV Shows, Movies, Photos, etc. And you can define other types (like home movies, music videos, etc.). This is important because the sources you use, or don't use, to get the metadata for the media is dictated/selected in the server program. Having a usable media library means you need some way of categorizing the media. Getting thumbnails, brief descriptions of the TV show, for example, make it hugely more usable when deciding what to watch.

So, just convert your CDs and DVDs and it's all digitized and ready to watch, eh?

Not really so easy...

Converting CDs should be simple--rip them into MP3 or FLAC or whatever your media server/transcoding allows (something I had to discover). Some fair amount of my music collection was already in MP3 format, though in varying degrees of  bit rates and I wasn't going to go back and reconvert what I already had. I only had one PC with a CD/DVD/BluRay drive, so conversion was going to be one disc at a time. I batched up my CDs. I experimented with using iTunes at 320kbps and a program under Ubuntu to rip into FLAC.  FLAC takes hugely more space, so though I wasn't cramped for space, the prospect of tripling (or more) the file sizes by using FLAC didn't seem (for me) to be worth the disk space, since my hearing and computer speaker system isn't able to allow me  to discern the differences. Now mostly finished, I have about 22,000 music tracks.

Photos- My photo library is still a bit of a mess because it's an accumulation of digital photos from 1998 forward and I have about 50,000 photos. Organizing this will definitely be a challenge due just to the sheer number of photos and the multiple ways you might want to organize them.

DVDs and Blu Rays- Could be complex to convert because they are (usually) copy protected and not just a single file as a music track or photo, and there can be features segments (directors cuts and the like) on the disc, along with the main video. For my personal library's use, I converted them using a variety of tools, with Blu Ray being the most time consuming. Due to their enormous digital content (which is what makes the quality what it is), they also produce a huge file size. One Blu Ray produced a 12+ gig file. So I had to do a second conversion using Handbrake to compress the file size down to something more reasonable (usually  4-6 gig for Blu Ray and ~600 meg to 1.5 gig or so for DVDs). The process starts by ripping the Blu Ray, which isn't something Handbrake does. So there's another tool. Then there's the decision about the file 'container' type- should they be mp4, mkv , or something else? What file types would the media server program support? Is there a file type that optimizes picture quality and file size? Nothing's simple.  I don't know the answer. I do know that due to the enormous amount of data on a Blu Ray, it takes a lot of time per disc to rip and re-encode the file.

For the most part, I decided to stick with Matroska (mkv) as it is an open standard container format for video/audio).  Sometimes the conversion didn't go well and I ended up re-ripping and using mp4 instead.  Nonetheless, I've found that Plex seems to support avi, mp4, mkv with transparency (as to format) when viewing it on the device. Whether the file requires transcoding seems to depend on the client used (PC, game console, phone, tablet, etc.) rather than the file type on the server.

Then to organize your media within Plex and have the proper metadata applied, so that the thumbnail artwork and descriptions align with the media, you need to follow their naming conventions, which (mostly) work. But sometimes it doesn't.  If you do everything according to the guidelines and your media fits within those, great.  I had questioned whether audiobooks would be supported, but they seem to be.

The support community within Plex is active and, I've personally found to be, very good, though you obviously need a certain level of technical knowledge to perform some of the things to get your Plex server up and accessible.  To their credit, the team at Plex have made it generally pretty easy. I bought a PlexPass to show my support for the project. Following the threads on the forums though, reveal that not everyone is up to some of the basics of Plexdom. In order for your content to be accessible outside your home, you may have to dig under the covers in your router to do some port forwarding and fixing some IP addresses. My router doesn't do UPnP, so both for my Synology server as well as the Plex server, port forwarding was a manual process.

I initially set up my Plex server on a Windows7 laptop (quad core i7, 8 gig RAM, 1tb hard drive), but due to it's consistent instability (none of which was due to Plex, it was unstable before all this). Then as my library started to expand, I realized I had to go with something more substantial. I could have added external USB hard drives, and had a number of those already. But the jumble of hard drives that would have ensued wasn't appealing. So when I bought the Synology, it was also intended to serve this purpose. But I wasn't sure the Synology NAS was up to transcoding, as it's real purpose is a file server and doesn't have a heavy duty processor and lots of memory (I upgraded my Synology to 4 gig RAM, from 2 gig). So I decided to move to a Mac Mini. This brought it's own set of challenges (mostly my learning, moving from a largely Windows background),and naturally, more money.

It all works now! But, how well it works depends on your download speed, the server's upload speed, whether it's on your local network, and then whether it's a wired or wireless connection, the client (what you are watching on), and whether (and how many) others are watching at the same time (if you've shared your library).

The move to the Mac Mini is the next part of the story.