I’m in the middle of a large project (for one person), and as part of it I wanted to host some CRTC documents on the project’s web server to link into. You see, if you’ve ever been involved in one of the CRTC’s public notices you’ll know that there are literal deluges of documents, many of which are zipped together. For the purposes of disseminating documents over email this works well – it puts all of the documents from say, Bell, into a single zipped file – but makes a user-unfriendly structure of linking to: expecting casual reader to link to zip archives is unreasonable. Given that as part of this project I do want to facilitate ease of access to resources it’s important that users can link to the documents themselves, and not zip archives.

While I pay attention to copyright developments in Canada and abroad, and have strong stances on how academics and the Canadian government should licence their publications, I’m not a lawyer. I do, however, know that government documents in Canada are governed by Crown Copyright – unlike in the US, the Canadian government maintains copyright over its publications – and thus I wanted to check with the CRTC if there were any problems hosting documents from their site, including those presumably under a Crown copyright such as the CRTC’s decision.

Today I spoke with someone at the CRTC and received word that I could rehost the documents, without any problems. They were ‘in the public domain’ and so I could do with them what I wanted. I was pleased to hear this, as I wasn’t sure if the Reproduction of Federal Law Order would apply to the filings of private companies – it should apply to the CRTC’s decision itself, but as the order is written it’s ambiguous (to me) where and how private filings ‘fit’. In any case, I’ll take the CRTC’s word and be pleased that their public notice filings, including the filings by private corporations, are ‘public domain’ works and not governed by crown copyright. In case someone is wondering why I even bothered checking with the CRTC, given that my intended uses of the works arguably fall under under the research and review criteria of Canada’s fair dealing provisions, I expected that my use was legitimate but wanted to check with some lawyers that I was actually in the clear. I didn’t want to have a project go live, only to receive lawyer-grams from the CRTC or private companies! The CRTC has lawyers, and I thought it’d be a decent idea to draw on their expertise. Had I gotten a baffling response (e.g. no, for copyright reasons you cannot host anywhere else!) I’d have gotten a second opinion.

So, that’s fine and good: I can use the documents. It was the rest of the conversation that was particularly interesting, and more than a little disturbing.

I learned that the CRTC occasionally removes documents included in public filings and, far more significantly, sometimes changes the actual documents themselves without notifying anyone, not ever parties who were involved in the public notice. Such changes are made to correct misstatements of fact, to redact content already on the record or make available incorrectly redacted content, and so forth. This has far-reaching implications: closed public notices can have documented modified, where such modifications could potentially have affected public awareness of the notice as it was ongoing. In the case of some ‘corrections’, this could be incredibly important: what if in PN 2008-19 (and this is entirely hypothetical and meant as an example: I have ABSOLUTELY NO REASON TO BELIEVE THIS IS THE CASE) it turned out that one of TELUS’ redacted sentence was made available to the public, and it stated that TELUS was in early consideration stages of using Deep Packet Inspection in their networks? Such a ‘correction’ would have substantial effects on the arguments put forward by various civil advocates, and would render someone who was not involved with the public notice as it was ongoing very confused about the apparently contradictory filing between (in this example) TELUS and public advocacy groups. Thus, not only do these secretive changes risk contaminating later research, but it could also undermine public confidence in the public notice process itself.

The other very interesting copyright-related item that I learned was that, while hosting the documents isn’t a problem, were I to scrape and do a check-sum between what I have hosted and what the CRTC hosts that would be considered a copyright infringement. The gentleman I spoke with professed not understanding how that would be a copyright issue, and was just passing on the message from the lawyers, but it’s incredibly bizarre. In effect, it states (to me) that I can host but cannot check to guarantee that what I’m hosting is ‘the most accurate/recent version’ unless I want to eyeball the documents and look for ‘corrections’ that may or may not be announced anywhere in the document or public notice website itself. From a government transparency point of view, this is deeply concerning: members of the public, if aware of the potential for relatively secretive changes to public filings, cannot automate any system to watch for such changes without running afoul of lawyers. The resources thus required to ‘check up’ on the CRTC would be enormous for the poorly funded civil advocacy groups and members of the public (neither eyeballs or time are in abundant supply!). Moreover (and again, I am not a lawyer!) in my reading a check-sum or something like it is required to actually comply with Canadian copyright law. The law,

authorizes anyone, unless otherwise specified, to copy federal legislation, statutes, regulations, court decisions and tribunal decisions without the usual restrictions that govern Crown copyright materials, provided that one is careful to ensure the accuracy of the materials reproduced and that the reproduction is not represented as an official version. (emphasis added)

Given that I’ll be hosting documents, to ‘carefully ensure the accuracy of the materials reproduced’ aren’t I required to set up some automated system, given that the CRTC can potentially just change or remove documents without any public notification or transparency? Does this mean that compliance requires me to manually check on a daily/weekly/monthly basis that all the files on the CRTC’s webpage are exactly as they were when I first copied them?

Admittedly, these changes to documents in public notices are supposed to happen ‘fairly rarely’ and there isn’t any reason to expect that the filings for PN 2008-19 (which is what I’m interested in for this project, right now) are going to change. It’s possible that there was miscommunication as a result of interjecting an intermediary between myself and the CRTC’s lawyers. I’m very happy that I can host the filings for PN 2008-19 and that all of the items in that filing (including the CRTC decision) are apparently ‘public domain’ and thus outside of crown copyright. Those are all great things and I appreciate that the CRTC was fairly quick in getting back to me (it took about 4 business days). I’m far less impressed with secretive changes happening to public filings, and am disturbed by the position that scraping for check-sum purposes would somehow violate copyright.

I’m not a lawyer, and I’ll be following this up with the CRTC to try and get additional transparency into what’s going on. Hopefully there was just a miscommunication; I would understand if regular scraping was a problem because it could bog down their servers, and that on that basis they could drum up DDOS charge or something, but to construe server access to guarantee I’m hosting the most up-to-date documents with a copyright violation is mind boggling, and screams of misuse of copyright to this non-lawyer.

Other posts you might be interested in:

1. Update: CRTC PN 2008-19 Filings
2. Update: CRTC PN 2008-19 ISP Filing Summary Document
3. Summary: CRTC PN 2008-19; ISP Traffic Managment in Canada

There is a metric ton of cash that’s being poured into eHealth initiatives, and to date it doesn’t appear that governments are recognizing the relationship between copyright law and eHealth. That makes a lot of sense in some ways – when most of us think ‘medicine’ and ‘doctor’ we think about privacy as one of, if not the, key issues (while, other than hopefully curing whatever is making us ill!). In this light, we wonder about the security of databases, the willingness of healthcare providers to limit access to records, and so forth. People in Canada are worried enough about privacy that, on the Ontario Government’s eHealth Ontario site, ‘Privacy and Security‘ are front and center as a main link on their homepage. When we turn to British Columbia’s October 23, 2009 Heath Sector Information Management/Information Technology Strategy and search for ‘privacy’ we see that the term appears on 18 of the report’s 55 pages. Moving over to the Ontario Information and Privacy Commissioner’s May 2, 2006 presentation on health information and electronic health records we, again, see emphases on the privacy and security concerns that must be posed alongside any movement to massively digitize the healthcare infrastructure.

What we see less of in the eHealth debate are the prevalent dangers accompanying threats to cut citizens off of the ‘net as a consequence of copyright infringement. It’s this issue that I want to briefly dwell on today, in part to start ramping up some thoughts on the wide-ranging effects of three-strikes laws that are starting to be adopted and/or seriously discussed in various jurisdictions around the world.

To give an overview, three-strikes laws in the copyright context are generally presented as a way of curtailing copyright infringement. Often acting under the assumption that a downloaded copy of a file is the equivalent of a lost sale, major content and rights holders insist that they are losing billions of dollars per year to Peer to Peer (P2P) filesharing. While I will note that this is a relatively insane equivalency (it is largely like arguing that every person who opens a book in a bookstore, reads it for about 10 minutes, and then puts it down constitutes a ‘lost sale’ – for more on this read Doctorow’s Ebooks: Neither E, Nor Books), there are lots of great articles you can read that deal with this issue and so I’m not going to dip my feet into that argument here. You might ask how it is possible to identify copyright infringing work, and one of the ways of doing so is through specific implementations of Deep Packet Inspection (DPI) appliances by Internet Service Providers (ISPs). Virgin Media is trialling a system that will generate a copyright infringement index (though won’t identify particular individuals who are infringing on copyright) and DPI vendors such as iPoque have already produced equipment that is designed to identify infringing file transfers in realtime.

Three-strikes laws have a common, general, format: after an individual is found, or believed to have been found, infringing on copyright three times they are forcibly disconnected from the Internet by their ISP either at the behest of the government or content holders. The particular legal structure that facilitates these ejections from the Internet differs – sometimes there is a presumption of innocence and requirement that infringing use is proven, whereas in other systems accusations alone suffice – but the common end is the same: during an era when broadband is seen as a key to performing job searches, gathering academic research, developing knowledge about our illnesses, and discovering new cultural artifacts, copyright holders want to insist that their intellectual property rights should be foregrounded and other social goods put to the back of the line.

This bring us to the question posed by this post’s title: “Will copyright kill eHealth?” What happens when, after investing billions of dollars in ‘revolutionizing’ the current health system so that individuals are ‘empowered’ to access their health records, governments and their citizens realize that they are in a digital wasteland, where accessing health records is dependent on good copyright-related behaviour? When I’m tasked with absolutely securing my wireless network so that a war driver can’t access my network and download some pop track, does this mean that I should sign up for expensive third-party services to guarantee access to my ‘newer and better’ health records and other government services? To alleviate these anxieties, perhaps I’ll be able to pay a small monthly fee to my ISP and they will, on my behalf, block anyone on my network from accessing potentially copyright infringing work from non-sanctioned repositories so that I can participate in the new digital economy.

I’m not suggesting that health authorities are necessarily experts in areas of copyright, network management, network surveillance, or data transactions. Typically, they’re not, and citizens don’t expect their M.D. to understand the ins and outs of file transfer protocols, file signature analysis, or data packet analysis. While I’ve suggested (recently, no less) that it is important to consider the particular impacts of certain ‘high-functional’ technologies, such as deep packet inspection, we do still need to step back occasionally and think of some of the possible impacts that high-functioning technologies and the surrounding basin of law might have on the delivery of core, newly digitized, government services. I don’t want to live in a world where my ISP has a real market incentive to ’sell’ me the equivalent of copyright-infringement ‘insurance’ on a monthly basis so that I can view the digital records that governments and corporations retain about me. I actually don’t think that ISPs want to live in this world either; in such a world they would be placed in a position of liability upon failing to prevent me from accessing infringing material! It’s because of the wide-ranging possibilities of network intelligence and the laws around it that research into network intelligence and security is so interesting – with the Western transition to the digital, and the ability to watch and impact digital flows, the role of the ISP will only become more and more significant to citizens’ daily lives.

Who can we turn to in the event that some kind of a three-strikes law becomes manifest in a Canadian context? It’s entirely possible that the inspection of data flows for copyright infringing material might be ‘privacy protective’ – privacy might be built into the infrastructure by design per the governing ethos of the Information and Privacy Commissioner of Ontario – and this suggests that the language of privacy may be insufficient to really limit the challenges of any three-strikes law. I have similar worries about the ability for consumer protection laws to effectively limit the negative consequences of a three-strikes law (though Canada does have one of the world’s forefront copyfighter’s on the people’s side), and if we require judges to hold real cases before individuals have their digital lifelines terminated then court costs over what are (likely) just people ‘browsing content’ will be exorbitant. The transition to electronically delivered content has (supposedly) been devastating for how the major content owners have been able to turn profits, but we need to go further than just say they need to develop better models instead of suing people. We need to ask this: should government be developing laws the prop up questionable current-day business models at the potential expense of wasting billions in public infrastructure upgrades, or should government be taking a longer view of things and start siding with both citizens and their own allocation of infrastructure dollars? I worry that if government doesn’t more prominently side with themselves and their citizens, we’ll see eHealth and other eGovernment ventures die under the knife of copyright reform and protection, and that would be a tragic shame and absolute waste of citizens’ tax dollars.

Other posts you might be interested in:

1. Three-Strike Copyright
2. Virgin Media to Monitor Copyright Infringement
3. Three-Strikes to Banish Europeans and Americans from the ‘net?

Over the past little while there has been considerable attention focused on Virgin Media’s decision to trial Detica’s CView copyright monitoring system. This system uses Deep Packet Inspection (DPI) technology to identify data protocols and likely files that are being transferred in order to generate a Copyright Infringement Index (i.e. a ‘Piracy Index’). As outlined by Detica, CView will let ISPs work with content creators to determine whether ISPs providing content through their portals lead to reductions in ‘infringing’ transfers of content through P2P file sharing.

The story about Detica’s involvement really broke with Chris Williams’ piece over at the Register entitled, “Virgin Media to trial filesharing monitoring system.” In the piece, he recognized that the trial will encompass roughly 40% of Virgin’s customers, that the aim is to measure overall levels of filesharing rather than identify individual customers, and (at least initially) will focus on music. After I read the piece, I send some questions off to Detica and posted them (“Virgin to Use DPI to ID Copyright Infringement“) based on my reading of Williams’ piece and Detica’s consultation paper, and shortly thereafter followed up with Detica’s responses and thoughts on CView and privacy infringements (“Update to Virgin Media and Copyright DPI“). Between the posting of my questions, and the response from Detica, Richard Clayton had a meeting with representatives from Detica and posted the information they released to him over at Light Blue Touchpaper in a posting “What does Detica Detect?” The Register was also able to get face time with people working at Detica, leading Williams to produce his second piece “Spook firm readies Virgin Media filesharing probes.”

In the rest of this post, I want to pull together the information that has come to light so that we can get a better picture of what is known about CView. As such, this is very much a summary rather than an analytic post; hopefully I’ll have time to delve the information more critically in the near future.

How does CView integrate with the ISP network?

In the interview with Williams, Dan Klein of CView acknowledged that the CView appliances are expensive enough that ISPs are unlikely to purchase very many of them. The system,

starts by using fibre taps to pick off traffic from an appropriate part of the ISP network. They use a fibre tap rather than “port mirroring” to make it easier for the ISP to be sure that they won’t disrupt any traffic. The links that they monitor need not be carrying all of the ISP’s traffic — they merely hope that it will be a statistically significant sample.

The raw traffic is then sent to the CView box, which can handle multiple 10Gbit links. The first stage of processing is in hardware (FPGAs), then software takes over. The “external” endpoint identity is discarded and the “internal” identity is encrypted using a key that is not made available outside the box (ie: the intent is to make the customer “anonymous” but to be able to link different activity from the same source). (“What does Detica detect?“)

Put in other words, the Detica CView system engages in a passive, offline (as opposed to inline) analysis – the traffic is split (i.e. mirrored) from the ISP network so that consumers don’t experience any meaningful impact on their speed, if any impact whatsoever is even felt.

What does CView detect?

The Register, in their December 7, 2009, article revealed that eDonkey, Gnutella, and BitTorrent were the protocols that were to be inspected by the CView. At the moment, the appliance is geared to examine for music files, but the original Register piece raises questions of whether or not it will necessarily be limited to just music files in the future.

How does CView perform detections?

After splitting the traffic into the CView appliance, it examines data traffic to determine if data is being carried along one of the three aforementioned file sharing services. Even when encrypting your data traffic, it is often possible to identify the protocol used based on the cleartext data that precedes the encrypted flow. It should be noted that the most recent Internet Evolution test of DPI provided test results confirming this capacity to detected encrypted P2P flows. Detections are performed passively, out-of-line from the ISP’s traffic. When data traffic is identified as being P2P a content field is generated with the below information:

• the encrypted (and thus anonymised) customer identity
• the type of P2P protocol
• the content identifier value
• the file size
• a timestamp

Where the P2P flow is encrypted, while a record is generated no data can be entered into its fields. In addition to this information, the CView appliance will generate an ‘acoustic fingerprint’ from the file – this is, perhaps, the ‘content identifier value’ that Clayton notes? – and then passes this information along to a separate statistics box that will identify whether the P2P file is copyright infringing.

What about anonymity?

Of course there are worries that a system like CView could be used to rapidly identify the copyright infringers that are operating on a particular ISP’s network. Given the information provided by Detica, the company certainly is trying to secure the anonymity and identificatory privacy of ISP customers. Specifically,

IP addresses are anonymized at the source/DPI device using a pseudo-random replacement algorithm, which also entails ignoring the external IP addresses. The key generation system is managed automatically by the device (and thus an ISP can’t muck around with the system), and keys are periodically cycled and redistributed. The keys are never made available outside of the device, and once a set of keys for a given time period are discarded they cannot be recovered – the process is irreversible. On this basis, we can argue that no subscriber ID is associated with the randomized replacement algorithm, there is no way to associate a subscriber ID with the pseudo-random number after the fact, and as such the anonymization system should serve its purpose. (“Update to Virgin Media and Copyright DPI“)

Richard Clayton maintains, as I do, that CView is employing DPI in a manner that addresses individual privacy and data protection concerns though is mindful of the possible RIPA issues. Specifically, he writes:

I should also address (especially given the huge fuss over Phorm) the rather important question as to whether the system is lawful to operate? Please note thatIANAL, but I’ve studied their writings in this area a fair bit…

The design as explained above seems to address issues of privacy and data protection (amalgamating statistics and discarding identifiers is a sound technique for jumping these hurdles). But there is then the vexed question of illegal interception. The system does “wire-tapping”, that’s obvious, but the criminal offence is called “interception” and that is carefully defined within the Regulation of Investigatory Powers Act 2000. I expect that Detica would wish to argue that there is no interception because no content is seen by any humans… however, spitting out the file identifier might in itself be sufficient to infringe. It may take some case law before anyone can say for sure. (“What does Detica detect?“)

It will be interesting to see whether or not CView faces the same calibre of public outrage as Phorm did; Phorm ran up against Alexander Hanff (amongst others) but Hanff has recently noted his inability to work as a privacy advocate ‘full time’ this round as he did against Phorm. Admittedly, I see Phorm as engaging in different activities as Detica, but the emphasis often placed against Phorm (as I read things) was DPI first, and behavioural advertising second. That might, admittedly, be a coloured reading on my own part. Regardless, I’m sure that Detica’s PR staff is breathing some small sigh of relief that they won’t be dealing with Hanff full-time.

What is the utility of CView to ISPs?

Detica is promoting CView as a way for ISPs to establish an ‘index’ of copyright infringements. As noted by the Register,

Perhaps most importantly, at least at first, CView will measure how the overall level of copyright infringement via peer-to-peer networks responds to Lord Mandelson’s letter-writing campaign. If the Digital Economy Bill is passed in what remains of this Parliament, those observed by rights holder groups sharing copyright material could start receiving statutory warnings in the post from their ISP as soon as April.

A year later a system of “technical measures” – bandwidth restrictions, blocked protocols and disconnections for the most persistent – imposed on ISPs by Ofcom, is likely to follow. If successful in trial, CView will allow Virgin Media to monitor how its customers respond to the regime, although it will not be involved in idenfiying infringers. (“Spook firm readies Virgin Media filesharing probes“)

Richard Clayton notes that it might be the case that a small number of files might be ‘incorrectly’ identified as infringing, but such identifications are likely small enough to be inconsequential. The worry, of course, is that minor tweaks might turn CView into a “first-class monitoring system” that can be used to identify and target individual users. While an injunction would be required for this, the pushes for such injunctions in the EU mean that this is something that must be kept in mind, even though UK media conglomerates have not previously sought such injunctions.

What isn’t (totally) clear to me

I think that we’re developing a pretty good understanding of what the Detica system entails, as well as its characteristics ‘out of the box’. I’m still unclear about how the ‘audio fingerprints’ are taken – I assume (based on Occam’s razor) that based on what has been released to Richard that hash-based, rather than fingerprint-based, methods of analysis are being performed but can’t be totally certain. (Note: fingerprinting can be used to detect infringement where only a fragment of a file is identified as infringing, as in a mashup that includes a second or two of a song, whereas a hash-based analysis would only examine the totality of the file, as in a .mp3 file of Madonna’s ‘Like a Virgin’.)

The other, fairly major, element that isn’t clear is just how easy it is to ‘tweak’ the CView system as Richard suggests is possible. If we’re talking about a firmware update, that’s a fairly low cost with potentially very major functionality changes to the device, and would run counter to the assurances provided by Detica to Richard, the Williams, and myself that the system is designed to provide anonymity. On the other hand, if it would take a hardware modification, then the infrastructure, manpower and capital expenditure costs to ‘upgrade’ the device might alleviate the drive for ISPs to implement a genuinely granular user-identification system. Function creep with these devices, of course, is a real worry – it would be great for Detica to clarify how these ‘tweaks’ are technically possible as part of their ongoing efforts to be transparent.

Other posts you might be interested in:

1. Update to Virgin Media and Copyright DPI
2. Virgin Media to Monitor Copyright Infringement
3. Will Copyright Kill eHealth?

Recently, I’ve heard back from Detica about CView and wanted to share the information that Detica has been provided. CView is the copyright detection Deep Packet Inspection (DPI) appliance that Virgin Media will be trialling, and is intended to measure the amount of copyright infringing files that cross Virgin’s network. This index will let Virgin determine whether the content deals they sign with content producers have a noticeable impact on the amount of infringing P2P traffic on their network. Where such deals reduce infringements, then we might expect Virgin to invest resources in agreements with content producers, and if such agreements have no impact then Virgin’s monies will likely be spent on alternate capital investments. I’ll note up front that I’ve sent some followup questions to seek additional clarity where the answers I received were somewhat hazy; such haziness appears to have been from a miscommunication, and is likely attributable to a particular question that was poorly phrased. Up front, I will state that I’m not willing to release the name of who I’m speaking with at Detica, as I don’t think that their name is needed for public consumption and would be an inappropriate disclosure of personal information.

The key question that is lurking in my own mind – if not that of others interested in the CView product – is whether or not the appliance can associate inspected data flows with individuals. In essence, I’m curious about whether or not CView has the ability to collect ‘personally identifiable information’ as outlined by the Privacy Commissioner of Canada in her recent findings on Bell’s use of DPI. In her findings, the Commissioner argues that because Bell customers’ subscriber ID and IP address are temporarily collated that personal information is being collected that Bell does collect personal information. In the case of Bell, this didn’t mean that they had to stop the collection, but that they had to adjust their privacy policies to reflect this collection (though it should be noted that any such association and collection will happen, with or without a DPI appliance, because Bell always associates a subscriber ID with dynamically assigned IP addresses).

Now, this means that my examination of the CView system and consideration of privacy is different from those approaching the system from the stance of Regulation of Investigative Powers Act which might be seen as putting me off-side of some privacy advocates. I don’t necessarily have an issue with that, and in fact think that strong, well meaning discussion amongst the privacy community can be quite healthy – different levels of analysis and approaches are called for when facing particularly novel technological systems, and expecting a lockstep approach of these technologies and their accompanying politics is somewhat absurd. For my purposes, I’ll simplify things and identify a privacy infringement (for my purposes, if not those of RIPA) as entailing:

1. A collection, processing, storage, or analysis of data that is associated with an individual, or a very specific set of individuals;
2. A case where whatever is collected, processed, stored, or analyzed is done so to influence the individual, or specific set of individuals, in a particular and reasonably direct manner;
3. An instance of data anonymization where there is the strong likelihood that such anonymization is either intentionally compromised or unlikely to be effective.

In terms of the CView system, let’s first address the concern of anonymization. Specifically, we have to ask how stringent the anonymization system actually is. When I asked Detica about this process, they informed me that because the CView device is intended to produce a Copyright Infringement Index (aka the ‘Piracy Index’) by evaluating the overall filesharing on a network that identity information isn’t required for this objective. IP addresses are anonymized at the source/DPI device using a pseudo-random replacement algorithm, which also entails ignoring the external IP addresses. The key generation system is managed automatically by the device (and thus an ISP can’t muck around with the system), and keys are periodically cycled and redistributed. The keys are never made available outside of the device, and once a set of keys for a given time period are discarded they cannot be recovered – the process is irreversible. On this basis, we can argue that no subscriber ID is associated with the randomized replacement algorithm, there is no way to associate a subscriber ID with the pseudo-random number after the fact, and as such the anonymization system should serve its purpose. Of course, there is a concern that there are no such things as anonymization processes – as noted by Paul Ohm – but I think that a more technical analysis of data logs would be required to figure out whether or not we could make the push that Detica’s system is a failure. At the very least, they appear to be making a real effort in keeping data sets anonymous and doing what they can to prevent privacy infringing behaviour.

One of the questions I posed to Detica, which was related to CView identifying copyright infringing files, went as follows:

“…what method is used to identify content. Is Detica using a file hash-based identification process or fingerprinting system? I ask because broadly identifying protocol alone would render any analysis of P2P data traffic as inherently infringing somewhat problematic, given that P2P is also used for legitimate file transfers.”

The member of the company I wrote to admitted that they couldn’t go into the specifics of how the system performed identifications for commercial reasons – this is normal when dealing with what are effectively corporate secrets – and thus couldn’t speak to their system using either fingerprinting or hash-based analysis. They did say, however, that the system is conservative, insofar as it makes its assessments based on assumptions that transfers are legitimate unless there are reasonable grounds for determining otherwise. As I read/translate this statement, it says to me that rather than classifying all P2P traffic as infringing, the system only flags infringing content as that which can be matched against its index of infringing files. Whether this entails fingerprinting (where only a fragment of a file is identified as infringing, as in a mashup that includes a second or two of a song, instead of the whole file, as in a .mp3 file of Madonna’s ‘Like a Virgin’), however, is unknown.

Detica’s responses maintain that their CView system is deployed in a passive mode (which is expected), and I’ve asked for clarification about whether or not it rests inline or offline – whether the appliances will perform traffic analysis in real time inline with the flow of data passing through the ISP’s network, or in a delayed fashion that sees the data traffic ‘offloaded’ out of the ISP’s network. I expect that it is a passive, inline appliance, but we’ll see. The company does maintain that “there is no persistence of any analysed content – Detica CView(tm) is a measurement system so could not be used as an evidence collection mechanism.” This means that the DPI appliance cannot be used, as designed, to identify individuals trading infringing material online, and thus cannot be effectively used to enforce any three-strikes law.

Ultimately, given that CView is engaging in network-level intelligence, without correlating IP addresses with a unique signature or code, let alone a subscriber ID, I’m not certain that this system is necessarily ‘privacy infringing’ as it’s presently configured and deployed. Does this mean that it can be used to subsequently insist on deeper penetration and analysis of who is trafficking following the establishment of a ‘piracy index’? Quite possibly – the political ramifications of having quantifiable network intelligence are vast. One of the reasons why DPI appliances in general are so interesting is how they are wrapped up in the politics of net neutrality, privacy, and copyright. Despite their interesting intersection along the crossroads of digital issues, perhaps we need to develop an archetype to engage with these devices as follows:

1. What does the technology do, today? Does this constitute a privacy (or, preferably, constitutional rights grounded) infringement?
2. What can the technology do, tomorrow? In light of what it can do, how should we advocate for strong protections to prevent our concerns from arising, and channel the technology towards ‘good’ outcomes?
3. Ask the question ‘what needs to be put in place to ensure that the ‘good’ outcomes of tomorrow triumph over the possible ‘bad’ ones?’ and provide resources to achieve the good and avoid the bad.

This is a simple schema (and, actually, deserving a deeper analysis), but parallels what I’ve come to adopt over the past few months. It is critical that we analytically distinguish between temporal realities and futural possibilities, as well as between the issues of network neutrality, copyright, and privacy (among others) to develop sufficiently nuanced and complicated understandings and resolutions to the insertion of DPI appliances in ISP infrastructures. DPI is unlikely to go away; the aim now has to be to identify and proclaim ‘good’ uses of the technology and work to prevent the ‘bad’ uses from becoming prominent telecommunication practices.

Other posts you might be interested in:

1. Virgin Media to Monitor Copyright Infringement
2. Aggregating Information About CView
3. Copyright and the Blank Media Levy

Late last week The Register reported that Virgin Media is going to be trialling Detica’s Deep Packet Inspection (DPI) appliances to measure the levels of copyright-infringing file sharing that is occurring along Virgin Media’s networks. It’s important to note a few things right up front:

1. I have a request in to the company manufacturing these appliances, Detica, and have been promised responses to my questions. In light of this, I’m not accusing Detica or Virgin Media of engaging in any ‘privacy invasive’ uses of DPI, at least not at the moment.
2. The information that I’ll drawing on is, largely, from a consultation paper that Detica presented in late September of 2009.
3. This post is largely meant as a ‘let’s calm down, and wait to hear about the technology’s details’ before suggesting that a massive campaign be mounted against what might be a relatively innocuous surveillance technology.

With that stated…

Detica describes themselves as a “business and technology consultancy specialising in helping clients collect, manage and exploit information to reveal actionable intelligence. As the digital revolution causes massive amounts of data to converge with a new generation of threats, many of our clients see this as one of their greatest challenges.” Their CView DPI system is meant to let ISPs better identify the amount of copyright infringing work that is coursing across their networks, in an effort to give ISPs better metrics as well as to determine whether arrangements between ISPs and content providers has a significant, measurable effect on the transfer of copyright infringing files.

The consultancy piece that is provided by Detica maintains that their DPI system is meant to preserve customer privacy, though the lack of technical insight in the paper itself means that many individuals and groups are deeply concerned about the actual instantiation of privacy-protective measures (e.g. Twitter’s #virginmedia hashtag is filled with concerns and complaints). What I’ll do is outline the most relevant parts of the consultancy piece, and follow each part with the questions that were provided to Detica. I’ll also include one question set that I forgot to ask, and will be sending along once I get my response from Detica. First, what’s exactly is this DPI appliance, anyways?

CViewTM is a pre-built, secure service providing a statistically-significant sample of all illegal file sharing activity across an ISP network. CViewTM resides within the secure environment of an ISP network, operating in a “lights out” environment (i.e. without human intervention). It aggregates all of the individual “in-network” file sharing activity and performs analysis on this dataset with pre- defined statistical models to present ISPs and CPs with detailed reports of the volume and nature of the P2P activity by subscriber groups (achieved by clustering similar behaviour types).

That the technology operates without human intervention is a no-brainer; the company is selling a device that is meant to massively aggregate and analyze data traffic. If an individual human, or team of network specialists, had to watch the logs and then run their own calculations based on what the log revealed then the product would be a dud for the purposes that it’s being sold for. Something that I expect is unsettling for many, especially those concerned with the impact of imposing statistical behaviour sets on users, is that there are pre-defined statistical models that will report on users by clustering similar behaviour types. What I failed to ask in relation to this includes:

• What are these kinds of behavioural types?
• Can the ISP with a CView product in their network infrastructure alter how behaviour types are collated?
• When Detica suggests (earlier in the consultancy piece) that they will be creating a ‘piracy index’, is this index meant to be a standard that is controlled by Detica, or is it fungible so that ISPs can configure it to suitably engage with their own customer base and customer habits? If the latter is true, then doesn’t this suggest that a industry standard index is immediately jeopardized?

There are a set of four principles that are ingrained with the development of the CView devices themselves. The first:

anonymous data collection — all records collected from the network have their IP addresses strongly anonymised such that no reference to an individual can be made, even in conjunction with other ISP systems. No content data is recorded (e.g. URLs).

As pertains to records being ’strongly anonymised’, I want to know what, exactly, this entails. While Google claims to have ’strong’ anonymization for the IP address information that they collect, they only remove the last 8 bits of the IP address in their logs. Given that this comprises the last octet only, and each octet can contain the values from 1-255, Google’s technique lets a computer user hide amongst 254 computers at most. Google’s approach is juxtaposed against, say Microsoft’s, which deletes cookies and full IP addresses along with other identifiable information after 18 months. What, exactly, is entailed in Detica’s ’strong anonymisation’ process?

The second:

proportional to right to privacy — traffic is inspected to establish what the content is and the application being used, with no persistence of traffic data or identity information.

I presume that this means that there is simply an inspection of the content, a record or log kept concerning what is (and what isn’t?) identified, and then no efforts to store content streams offline. Is traffic inspected inline with the Virgin network, or is content being offloaded and subsequently analyzed ‘offline’? I fully expect that CView examines known protocols (which DPI appliances are generally capable of doing) but wonder what method is used to identify content. Is Detica using a file hash-based identification process or fingerprinting system? I ask because broadly identifying protocol alone would render any analysis of P2P data traffic as inherently infringing somewhat problematic, given that P2P is also used for legitimate file transfers (in Canada, our national news station, film board, and other government bodies are using P2P for the dissemination of public content, as an example), and there are substantial differences between the application of fingerprinting or hash-based systems (fingerprints might catch mash-ups, whereas hash-based targets full files). Further, when it is suspected that encrypted P2P traffic is crossing a network, does this constitute infringing traffic, non-infringing, or place a user in an entirely separate  behavioural category-type?

The third and fourth:

closed system — no traffic data or identity information is ever made available to a person. Traffic application data is produced by an entirely closed and automated “lights out” system. Appropriate hardware, software and process controls prevent intentional or accidental breaches of privacy (e.g. preventing access to the live system when data is being processed).

no feedback loop — none of the behavioural data collected can ever be attributed back to a person or drive action against an individual.

This appears to be a positive maneuver, though I would wonder how much access ISPs actually have to these devices: are they prevented from reconfiguring these devices, or offloading information to a SAN for their own analysis? Should an ISP demand it, is is even possible for these devices to disclose the traffic data or identity information of ISP subscribers and their related data traffic? How are updates performed to the device, and what would such updates comprise (e.g. would they update the protocols/files that are detected, or go so far as to modify the ‘piracy index’ as well and extend the ability to discretely associate infringing content with particular IP addresses)? Finally, given that different categories of users are being established, while the device cannot use behavioural data to target individuals, can it be used to target the groups that the device identifies?

As stated at the head of this post, I haven’t heard back from Detica, and until I do I’m refraining from decrying (or praising) this technology, in part because I’ve been aware of this kind of technology for some time: despite Detica’s suggestions, DPI manufacturers such as iPoque have included this in some of their devices for some time (also, and contrary to their consultancy piece, Canadian ISPs have been tracking P2P use for some time). Identifying and preventing the distribution of copyright infringing files, while certainly a problem for the P2P movement, would likely be read (in the UK) as complying with the recent ‘Digital Britain’ initiatives. If the technology genuinely provides some significant level of anonymity, and presuming that the ‘piracy index’ isn’t rigged in some manner (perhaps have it open-sourced?), then this could just be a manifestation of a company selling a very particular product to address a particular need for network intelligence in compliance with British Law. This isn’t something that is strongly desired by some parties – the ‘dumb pipe’ position is commonly adhered to by network neutrality advocates – but perhaps speaks to the real need to address the misconceptions about information services, and how they legally (at least in Canada and the US) differ from telephone services.

I’ll end this by noting that I’m less familiar with UK law and regulations; I don’t know RIPA in and out, and I’m not trying to justify the Detica appliance. Instead, I’m just suggesting that until more data is released that privacy advocates and network neutrality advocates alike should take a step back, take a deep breath, and wait for a little more information before letting loose the dogs of war.

Other posts you might be interested in:

1. Update to Virgin Media and Copyright DPI
2. Copyright and the Blank Media Levy
3. Aggregating Information About CView

I’ve been watching with some interest the new Artist 2 Fan 2 Artist project, recently started up by Jon Newton and Billy Bragg. The intent of the site is to bring artists and fans together and encourage these parties to speak directly with one another, without needing to pass through intermediaries such as producers, labels, public relations groups, managers, and so on. It will be interesting to see how the dialogue develops.

One of the key elements of the site that interest me the discussion of paying artists (and other content creators); how can we avoid demonizing P2P users while at the same time allocating funds to artists/copyright owners in a responsible manner. On October 5th, this topic was broached under the posting ‘In Favour of a Music Tax‘, and I wanted to bring some of my own comments surrounding the idea of a music tax to the forefront of my own writing space, and the audience here.

I think that an ISP-focused levy system is inappropriate for several reasons: it puts too much authority and control over content analysis than carriers need, puts carriers at risk when they misidentify content, and would make carriers (for-profit content delivery corporations) in charge of monitoring content without demanding consumers that pay ‘full value’ for content moving through their networks. This last point indicates that an ISP-based levy puts ISPs in a conflict of interest (at least in the case of the dominant ISPs in Canada). Another solution is required.

I’m a Canadian, and I don’t have an issue with a levy-system that works. In Canada, we are charged a levy on all blank media that is sold. The levy originated several years ago, and was meant to recoup losses from the copying of mp3s (and related audio files) onto disks. The levy is very small:

• $0.24 per unit for Audio Cassette tape (40min or longer);
• $0.21 per unit for CD-R Audio, CD-RW-Audio & MiniDisc;
• $0.21 per unit for CD-R, CD-RW (non audio).
• In 2009 the levy on CDs and MiniDiscs will rise to $0.29 (Source)

Arguably, fewer people burn mp3s onto disks than in the past – with the advent of cheap and portable storage media, media tends to find its way to mp3 players and similar portable (and slightly less portable) media environments. Rather than targeting ISPs (who, really, should function like semi-intelligent data piping networks, rather than smart content monitors) why not impose a small levy on mp3 players/consumer electronic storage equipment? When I see an iPod or something like it, and am told that ‘160GB of storage just isn’t enough’, I have a hard time believing that someone has paid the not-so-small fortune to fill it with music, TV shows, and other content. I argue that we should toss a levy on the device at the point of sale, and then have a mechanism where the money is blackboxed and distributed back to content owners.

Is blackboxing ideal? No, but distributing the funds collected can be improved by developing either a more broadbased sampling of what is being listened to (i.e. one that accounts for the long tail of P2P downloads, YouTube watches, etc) or ’smarter’ devices (i.e. ones that can read what is being played and then report this back to a mothership) to adjust the levy distribution in near-real time. As a consumer, the former is an acceptable shift in distributing the levy, whereas the latter strikes me as a gross violation of my reasonable expectations of privacy (regardless of whether it would cross an actual legal boundary). Let’s think through what this ‘tracking’ system might look like:

First, it won’t involve any deep packet inspection appliances, so we can dispel any worries associated with that particular technology. Instead, we get a few clever programmers together to develop an aggregation system that identifies the number of downloads of songs/albums/artists across a wide set of media environments (e.g. P2P sites, YouTube, MySpace, etc). Either the sites themselves or the aggregation system can include anonymization protocols that scrub data sets after initial collation, ensuring that the data can’t be traced to individuals/sites. Scrubbing protects consumers and has the benefit of mitigating some efforts to ‘game’ the system by demanding particular sites are included in the aggregation step at the expense of less favourable ones. It should be noted that I’m not seeing this system as one that tracks particular users but instead the number of downloads particular sites have for files. While this doesn’t generate 100% accurate picture of content transfers, it will capture the long tail better than the present Canadian levy’s evaluation metrics. Taking account of the fluidity of the P2P environment, it is important that the aggregation system is designed so that adding new sites to the scrape is relatively simple, and the system generally should be modular so that it can scrape new P2P platforms as they come available.The nice thing is that, once this system is paid for and set up, it’s largely self-running AND can be exported to different jurisdictions as needed. This system should be open source, so that anyone who is interested can take a look at how the algorithms calculate levy distribution in an effort to facilitate transparency and rigorous code-checking.

While there are certainly valid concerns around the organizational collection and distribution of these levy fees, I see a distinction between the value of a levy and organizational logics. In Canada, as an example, a key issue that I keep hearing between major players in the content industry is that the metrics used to identify popular artists is defunct – it needs to be broadened to capture the long-tail of media downloads. The Canadian situation sees (relatively) little overhead, and has distributed over $100 million dollars since the levy’s inception. This isn’t a small amount of money.

This model does recognize that labels will get some of the money out of the levy system should they own the copyright of the content being downloaded. There isn’t any reason why they shouldn’t receive some of this money from a (layman’s) legal perspective. If this is perceived as a problem, because it ignores the artist, then that’s a larger issue of copyright ownership and not a reason why the levy itself is unworkable. If artists retain their copyright, they get the cut of the levy fund instead of a label. If there is a strong desire to change how present copyright systems work (and I think that there should be) that doesn’t undermine the levy-system; we should reform rather than scrap the levy, and not let copyright/label issues motivate a refutation of the levy model.

Perhaps most crucially, the levy is not designed to capture the ‘full value’ of media – it doesn’t correlate one download to one sale. Instead, it recognizes that there is partial value derived from a download that is unlikely to parallel the same value that is demonstrated when someone purchases a CD, pays to download music through a content delivery system like iTunes, or attends a concert. If we take the stance that P2P is a means of facilitating interest (and partial value) for content then it is obvious that the levy-system is not enough, on its own, to create a full-value stream between artists and fans. Additional monetization schemes must be adopted so that, after deriving partial value from a download, fans (as distinguished from downloaders) can provide what they perceive as full value to the artist(s) in question. How this is done extends beyond a levy system – while important, I don’t want to work through my thoughts on this extension right now.

I want to quickly turn to some very broad, general, and tentative ideas of how the collection and distribution of a levy might work. This is most definitely a work in progress.

While there is a worry that the levy could increase as various parties who are not currently supported by the levy system try to muscle their way in, I think that the solution is just to set a particular (fixed) rate for the levy, and have it increased by X (where X=rate of inflation, or some other clearly identified, relatively autonomous metric that is outside of the music market), and then an organizational metric for distributing funds. Figure that every Y years the distribution metric could come up for organizational review, with the review board being composed of a balanced group of people (e.g. members of the non-profit’s board, artist and label copyright holders, hardware vendors, fans, etc). Where it appears as though the non-profit levy-distribution group is overstepping their bounds, a judicial review could occur (as happened in Canada, when there have been attempts to extend our levy to portable music devices).

In terms of actually establishing how much money the levy should collect per device, I’m inclined to set a very low levy that is based on a per GB/TB calculation of the storage media purchased. These rates should be automatically adjusted over time per a formula that takes into account drive size increases by reducing the cost-per-GB/TB so that the levy doesn’t get prohibitively high. The algorithm for this can be developed prior to the levy being restarted/instated to avoid hidden surprises and put in the public domain. Participation from a wide community should be invited, with a recognition that any formula set is a compromise position; there is no way that everyone is going to be happy. Consensus is highly to emerge between all the interested parties.

On the whole, what I’m trying to express is that levies are workable, partial, solutions. They do facilitate artists getting paid for their contributions to culture. They do benefit the consumer because it is clear why the levy exists and how funds are distributed without adding complexity (for the consumer) at the point of sale. Levy systems do not try to equate one download to one sale, and instead recognize the partial value of downloaded media. The proposed hardware-centric levy does not require ISPs to spy on content as it moves across their networks. Further, as I’ve proposed the aggregation of data points, this levy system does preserve anonymity and privacy.

Does this approach ultimately result in consumers paying a slightly higher price for blank media/devices? Sure, but I don’t think that’s anymore unreasonable than paying the trivially small environmental taxes already associated with the purchase of new computer equipment.

Other posts you might be interested in:

1. Virgin Media to Monitor Copyright Infringement
2. Update to Virgin Media and Copyright DPI
3. Analysis: ipoque, DPI, and copyright

When people are about to download content from the ‘net that is copywritten, many often ask ‘will I get caught doing this?’ For many, the response is ‘no’ and then continue to download that episode of Seinfeld or whatever. Given that there are so many people downloading, and that record companies in the US have claimed to have abandoned filing new lawsuits against individuals, then things (in North America) appear to be getting better.

At issue, however, is that filing lawsuits is big money, and in Europe especially it looks like Digiprotect has moved in to assume first-mover advantage. Digiprotect gets “the legal rights from the companies to distribute these movies to stores, and with these rights we can sue illegal downloaders. Then we take legal action in every country possible, concentrating on the places where such action will be profitable” (Source). They avoid demanding too much money from infringers, on the basis that few judges like the idea of imposing million dollar fines on individuals – usually opting for suits demanding in the vicinity of 500 Euros. This amount of money ‘teaches’ individuals and provides enough money to keep the employees paid. No staff member has a fixed salary – they are paid according to the ‘cases’ that are won. The actual method of determining the financial burdens are based on the business expenses, profit, and money to be distributed to artists. In effect, the company sets up a honeypot and then sues whomever it is profitable to sue.

On the one hand, you have to give the company some credit: this is a particularly innovative way of eking out an existence. Of course, various types of bacteria that eke out similarly interesting existences under rocks. This seems to be a extortion racket similar to what the RIAA has been running in the US for years; by not asking for absolutely massive amounts of money it is (almost) senseless for the individual consumer to hire a lawyer and put up a fight in court. The amounts of money are low enough that it makes more sense to pay Digiprotect and get them to go away. At the same time, however, it means that an individual who has been falsely identified (which is highly likely, given that Digiprotect performs their backwards trace using the IP address alone) is in a tough situation where they either fight the charge and build up their own legal costs, or pay out to avoid an escalation of costs. This isn’t just speculative theorizing: the BBC has an article that discusses that people who clearly haven’t infringed on copyright are being sued on a regular basis by Digiprotect.

There is no indication of how the IP address is maintained, what information the company links it with, how long data is stored, etc. In effect, it’s unclear whether the company is adhering to German data protection laws – I presume that by participating in a P2P session an individual is not ‘consenting’ to a collection of their personal information that extends beyond the mere technical requirements of sharing data. On this basis, I cannot see how it is reasonable for someone operating in a privacy regime that espouses the need to minimize data collection and retention, while maintaining consent, can let a company obfuscate their actual business practices that depend of the capture of personal information under the auspice of honeypot activities.

Further, since copyright holders are licensing the content to be distributed on P2P systems it is unclear to me how an individual is ‘infringing’ on content where distribution licenses have been secured; licensing content to be shared, and then suing individuals for taking advantage of that sharing seems obtuse. The companies that are using Digiprotect are aware that their content is being put on P2P networks; the aim is to ‘catch’ people and sue them after the company intentionally puts itself in a position of being ‘harmed’ by virtue of sharing their content. This is particularly snakey; companies claim that P2P damages their industry, and to prove their point have their own contend seeded on P2P networks and sue people as ‘proof’ of the harms caused by P2P. In some senses their actions remind me of skits where individuals throw themselves in from of cars, claim that the driver injured them, and then sue them for this harm that the individual put themselves in. I guess it’s a good thing that there isn’t a file sharing insurance packages – the might of insurance companies opposed to payouts would surely shut down companies like Digiprotect like they do claims submitted by people throwing themselves in front of cars.

Of course, companies like MediaSentry have been engaging in these sorts of actions for year, but the difference between them and Digiprotect are:

1. MediaSentry works for the RIAA, and thus is an instrument of the labels rather than a semi-autonomous profit-making company (MediaSentry makes a profit, but through far more direct relations with labels). There isn’t an attempt to keep MediaSentry at a significant arms length, whereas Digiprotect has their own network of law offices who are receptive to this technique, and just pay a portion of their suit-earnings to labels. The labels get a cut without having to ’smear’ their own images in court;
2. MediaSentry doesn’t have to worry about EU privacy laws

As for the second of those claim, I’d need more time than I can devote to break down all the differences between American and German data protection law, and then see if Digiprotect is in violation of law by developing databases containing people’s personal information without their consent. Given that Germany, at least, is far more sensitive to that issue it would be interesting to see how/if Digiprotect has gotten past that privacy block. More broadly, I wonder if Digiprotect is in any way worried about DPI appliances that are sensitive to copywritten files’ signatures as potentially endangering their operations? iPoque could be their worst foe in the sphere of competition; could DPI ’save’ consumers from this nasty business model where DPI appliances pre-emptively prevent the sharing of tagged copywritten files?

Other posts you might be interested in:

1. Newspapers: Effects of Closing their Content Ecosystem?
2. Copyright and the Blank Media Levy
3. Will Copyright Kill eHealth?

Universities in the US have been deeply burdened by the Higher Education Opportunity Act that President Bush signed into law last year. In particular, the Act require that “schools ensure they are doing all they can to combat illegal file sharing among students. The new rules, according to the wording contained in the legislation, requires institutions to develop plans to “effectively combat the unauthorized distribution of copyrighted material, including through the use of a variety of technology-based deterrents.” Schools must also “to the extent practicable, offer alternatives to illegal downloading or peer-to-peer distribution of intellectual property.” Any institute found to be non-compliant could lose federal funding” (Source).

To combat unauthorized distributions, technological solutions such as bandwidth shaping and traffic monitoring need to be implemented. Such solutions need to be integrated with advanced DMCA response practices. Of course, some of the companies that are being courted to meet these demands are those that incorporate DPI into their copyright ’solutions’. I’ve discussed, generally, how these technologies work on campuses from iPoque’s position when writing about one of the company’s whitepapers. In that post, I wrote,

In essence, ipoque argues that educational institutions should consider deploying DPI to limit their students from consuming bandwidth for infringement purposes, where that bandwidth is needed for other university business. This is an effort to normalize DPI by ‘teaching’ the educated elite that filtering is permissible even in educational environments. If such activities are permissible in a space of academic freedom, then surely filtering practices are permissible outside of these special environments!

The attempts to lean on schools to ‘regulate’ their students is a mode of governance meant to instill a particular, very American, conception of intellectual property and copyright into the hearts and minds of university students. Given the broad implications of American attitudes on these subjects, as well as on the topics of cultural growth and free speech, IT departmentsshould not be the business of deciding whether DPI should come to campus; students and faculty members who are indebted to academic freedom and best able to understand the implications of wide-spread filtering of Internet content should have the first, and final, say about whether these network appliances are permitted onto campus grounds.

I stand by my worry that pervasive analysis of students’ data transfers threatens to normalize persistent analyses of all data traffic, though in this case there is a federal directive that universities are (effectively) obliged to ‘teach’ students that copyright infringement is a ’serious’ offense. Long-term normative effects may be felt, though this assumes that students won’t quickly find ways to evade any DPI-based solution. Some methods of evasion might include using P2P systems that have proxy-capabilities baked into the systems to obfuscate destination and origin IPs while simultaneously encrypting traffic. Given that students are often at the forefront of counter-surveillance movements (when they impede activities they want to engage in) and the growing ease of using highly sophisticated P2P programs, I have real doubts that it is even possible for network admins to actually limit P2P should intrusive monitoring systems be adopted.

I really see a few things emerging from this act, widely, across American universities:

1. Money will be spent on anti-infringement technologies. Most technologies will be unsuccessful in stopping infringements, but will let schools continue to receive federal funding.
2. Students will quickly find ways around the technologies that have been deployed, effectively nullifying the effectiveness of these technologies.
3. As a result of how students get around these technologies, these same students might ‘get used’ to the idea that all of their data is analyzed and learn just how easy it can be to encrypt and proxy their traffic.

While some might see #3 as a positive (Christopher Soghoian has written about this some in the context of DPI), I worry that this just extends to ‘encryption wars’ that have been ongoing for the past two decades or so without actually addressing the social issues at hand. Do we really want to live in a world of such persistent surveillance? While some people might claim that they have nothing to hide, this doesn’t mean that you trust everyone or that you are willing to disclose how you operate to devices that are not, and will likely never be, perfectly accurate. 70-90% accuracy is great…until you need to spend thousands or millions of dollars to clear your name from an incorrect copyright infringement charge.

It really seems that America (and Canada too, for that matter) must think about surveillance from an ethical, rather than just a legal, framework. Fair Information Principles (FIPs) will ensure that companies meet certain criteria for engaging in surveillance, but do not necessarily ask whether or not they should be engaging in particular kinds of surveillance in the first place. The legal discussion is helpful, but not sufficient, if we’re to genuinely engage with and challenge some of the uses of data analysis/surveillance systems.

Other posts you might be interested in:

1. Analysis: ipoque, DPI, and bandwidth management
2. Deep Packet Inspection: What Innovation Will ISPs Encourage?
3. Will Copyright Kill eHealth?

This is a full draft of the paper on Twitter and privacy that I’ve been developing over the past few weeks, entitled ‘Who Gives a ‘Tweet’ About Privacy?’ It uses academic privacy literature to examine Twitter and the notion of reasonable expectations of privacy in public, and is written to help nuance privacy discussions surrounding the discourse occuring on Twitter (and, implicitly, similar social networking and blogging sites). The paper focuses on concepts of privacy and, as such, avoids deep empirical analyses of how the term ‘privacy’ is used by particular members of the social networking environment. Further, the paper avoids delving into the web of legal cases that could be drawn on to inform this discussion. Instead, it is theoretically oriented around the following questions:

1. Do Twitter’s users have reasonable expectations to privacy when tweeting, even though these tweets are the rough equivalent of making statements in public?
2. If Twitter’s user base should hold expectations to privacy, what might condition these expectations?

The paper ultimately suggests that Daniel Solove’s taxonomy of privacy, most  recently articulated in Understanding Privacy, offers the best framework to respond to these question. Users of Twitter do have reasonable expectations to privacy, but such expectations are conditioned by juridical understandings of what is and is not reasonable. In light of this, I conclude by noting that Solove’s use of law to recognize norms is contestable. Thus, while privacy theorists may adopt his method (a focus on privacy problems to categorize types of privacy infractions), they might profitably condition how and why privacy norms are established – court rulings and dissenting opinions may not be the best foundation upon which to rest our privacy claims – by turning to non-legal understandings of norm development, degeneration, and mutation.

Paper can be downloaded here.

Other posts you might be interested in:

1. Twitter and Privacy in Social Context
2. Twitter and Statutory Notions of Privacy
3. The Geek, Restraining Orders, and Theories of Privacy

I rely on other people to produce content for me to consume, and I reciprocate by providing my own content (via this blog, government submissions, submissions to alternative news sites, interviews on radio, etc.) to the public. I see this as a reciprocal relationship, insofar as anyone can come here and use my content so long as they abide by my creative commons license. Unfortunately, most advocates for newspapers would see what I do (i.e. blog, think publicly) as unequal to their own work. I’m just an amateur, and they’re the professionals.

One of my colleagues recently linked me to a statement that David Simon presented to Congress about the life or death of newspapers. His argument is (roughly) that bloggers and other ‘amateurs’ cannot be expected or trusted to perform the high quality journalism that these ‘amateurs’ then talk about online (Note from Chris: clear case in point, the critical analysis by journalists of the Bush administration and Iraq compared to bloggers. Oh…wait…). You need dedicated professionals who are professionally trained to generate consistently high quality and accurate content. At the same time, the for-profit model of newspapers has led them to cannibalize their operations for profit. Newspapers will perish if capitalism and the market are seen as ’solutions’ to the demise of newspapers, just as amateur culture and their appropriation of media will destroy content producers. Something must be done.

The thing is: I don’t think that ‘newsmen’ (as Simon uses the term), actually have a clue what they mean by ‘high quality’ news. At the very least they don’t have a regular definition of the term. There are some examples of what would constitute ‘high quality’ reporting,  but many tech journalism (just as an example)  amounts to no more than an advertisement. It’s pretty rare that you actually see the ‘indepth reporting’ that Simon is referring to, in part because good critical reporting might risk advertisers. Maybe regular ‘high quality’ reporting existed before the 90s, when I started reading the paper on an occasional basis, but I have my doubts.

I also don’t think that newspapers can put a genie back in the lamp. There is a lot of talk that papers need to lock up their content and charge online visitors (this is Simon’s suggestion). Rupert Murdoch is on record saying that his organizations will start locking up information within the next twelve months – “The current days of the internet will soon be over” (Source). I have my doubts that this is true (that the ‘net as we know it will change), or that his stratagem will be successful. I actively avoid sites that requires me to register, let alone pay, to read news. Don’t offer it at a fair (i.e. free) rate, then I don’t read. Papers have long survived on advertising, and the current ‘predicament’ facing most papers seems to be one of monetizing their content while at the same time their back-end costs should be plumetting. You don’t monetize content by hiding it away from the public, you do it by finding interesting revenue streams that surround and reinforce your content model. Why not work with an ereader company to sell a piece of hardware to subscribers that automatically downloads your paper plus related content on a daily basis, and then mine that data for better advertising to the individual?! Get rid of pulp costs, and replace it with digital hardware!

Search engines live and die by their ability to both index information and deliver it within an advertising laden search space. Search companies are, really, advertising companies. The ‘newsman’ stance that search means that individuals don’t read a whole paper, and thus lower the value of newspaper advertising is only true to a point. Are you selling a paper or are you selling stories. While your paper might be made more valuable as a result of publishing a host of very good stories, Search Engine Optimization (SEO) should be integrated into the very writing of the stories themselves. Aim for high SEO ratings, effective contextual advertising, and individual stories become more profitable given that they rise to the top of searches. Produce enough good stories and make it easy to subscribe using RSS, and you can rope in a customer for some time. This is what profitable blogs do – why, exactly, can’t newspapers do it too?

Oh, right. Because newspapers only hire ‘professionals’. Whenever I hear this phrase I’m thrown back to a president’s dialogue that the University of Guelph hosted a few years ago (link, though looks like video isn’t hosted anymore). In it, a regular theme between it and the subsequent dialogue was that good journalists are nosy, analytically bright, and liberally educated individuals. Pamela Wallin, now the Chancellor of the University of Guelph, actually commented that journalists emerging from journalism programs are rarely the Pulitzer winning journalists newspapers love to have. (At the same time, if journalists are taught to love search, learn SEO like no one’s business, they might become the lifeblood that gets enough attention to your paper that your Pulitzer pieces get seen by the public before they win the prize…)

Nosy. Analytically sharp. Liberally educated. Toss in a bit of ‘obsessive compulsive disorder’ and you’ve got the recipe that motivates many of the top-tier bloggers, and these individuals have made their media ecosystems work. Look at Ars Technica (I’m a techy – they’re a regular point of reference), Life Hacker, or Gizmodo. While each cover different facets of issues (and arguably Ars is the closest thing to a ‘real’ newspaper) the reason that they thrive is because they do a very narrow range of things very, very well. Gizmodo doesn’t often cover anything that doesn’t relate to gadgets or could produce a nerdgasm. Life Hacker doesn’t run political stories. These sites sit right between ‘mass media’ (i.e. needing to independently cover everything everywhere) and ‘the long tail’. It’s interesting to note that, very often, they will link out to sites (even competitors!) when those sites have produced particularly good work – they are actually better referenced than most ‘professional’ journalism’ pieces. I can only imagine what would happen if journalists had to link to their sources – I imagine that their ‘professionalism’ in many cases would melt away.

We live in a world where there is really no need to have 150 journalists tracking the same bloody story. One or three? Sure, I guess. But with the rising costs of travel combined with the decreases in revenue streams it is incredibly important to recognize that a strong newspaper depends not just on content but also on its community. By locking up information you are effectively throwing out a copyright gauntlet – ‘amateurs’ can’t use content without paying, and if they do use it without paying they’re likely to face DMCA and copyright infringement charges. This approach was tested by Big Media conglomerates in Hollywood, and has effectively alienated their consumers (who like the RIAA and MPAA, and by extension the companies who are associated with them?). I expect that similarly locking content away will sour people’s attitudes towards papers. Where will people turn when they feel slighted by ‘the professionals’?

Yeah, they’ll go to the amateurs. And you know what? This isn’t new. Jump back a century, and there were a lot of very cheap, very ‘amateur’ papers that would thrive for a short while and then vanish. Bloggers and sites like the Huffington Post will love the locking up of ‘professional content’ because it will drive people away from ‘professional site’ to ‘amateur sites’. About the only case where this won’t be the case is where you actually have real journalism being evidenced in almost every story (Economist, I’m looking at you). Most papers are not the Economist, or even approach its standards and caliber.

I do have to admit that I’m curious to know just how many ‘professional journalists’ these ‘amateur’ sites will pick up as they grow in the face of a content lockup, and whether these journalists will subsequently lose their ‘professional’ status and become mere ‘amateurs’.  It is, after all, where you work, as opposed to the caliber of your work, that indicates ‘professional’ or ‘amateur’ status, right?

Update/note: Nate Anderson has a great piece on the AP’s threat to sue those who simply ‘reproduce’ the news – note the distinction between facts and copyright in the dissenting views of the 1918 ruling.

Other posts you might be interested in:

1. Follow-up: Newspapers and Business Models
2. Study: Stolen Web Content Sees More Traffic Than The Original
3. Shield the Sources, Shield the Telecoms