{"@context":"http://iiif.io/api/presentation/3/context.json","id":"https://amiastreaming.aviaryplatform.com/iiif/br8mc8rk4z/manifest","type":"Manifest","label":{"en":["Johns Hopkins Digital Preservation Repositories for Research and Science"]},"logo":"https://d9jk7wjtjpu5g.cloudfront.net/organizations/logo_images/000/000/016/original/AMIA-Logo-17.jpg?1556650205","metadata":[{"label":{"en":["Type"]},"value":{"en":["Presentation"]}},{"label":{"en":["Coverage"]},"value":{"en":["Museum of Modern Art (Place of Recording)","New York, NY (USA) (Place of Recording)"]}},{"label":{"en":["Date"]},"value":{"en":["2015-05-08 (created)"]}},{"label":{"en":["Agent"]},"value":{"en":["Sayeed Choudhury,  Associate Dean for Research Data Management, Johns Hopkins University and Hodson Director, Digital Research and Curation Center, Johns Hopkins University (Speaker)"]}},{"label":{"en":["Publisher"]},"value":{"en":["Association of Moving Image Archivists"]}},{"label":{"en":["Description"]},"value":{"en":["\u003cp\u003eThe Sheridan Libraries at Johns Hopkins University have been storing and archiving research data for over a decade and have recently developed a data archive through initial funding from the National Science Foundation. Through work with the Sloan Digital Sky Survey and the development of the Data Conservancy, the Sheridan Libraries have developed a conceptual model for data management, a definition of preservation inspired by the Open Archival Information System (OAIS) reference model and a set of lessons learned.\u003c/p\u003e\r\n\u003cp\u003e \u003c/p\u003e\r\n\u003cp\u003eThis talk will describe the history of this data curation work and its culmination through the development of the JHU data archive. Additionally, the talk will feature challenges or opportunities related to data management that span different types of data from a diverse array of communities or organizations.\u003c/p\u003e (general)"]}},{"label":{"en":["Language"]},"value":{"en":["English (Primary)"]}}],"summary":{"en":["\u003cp\u003eThe Sheridan Libraries at Johns Hopkins University have been storing and archiving research data for over a decade and have recently developed a data archive through initial funding from the National Science Foundation. Through work with the Sloan Digital Sky Survey and the development of the Data Conservancy, the Sheridan Libraries have developed a conceptual model for data management, a definition of preservation inspired by the Open Archival Information System (OAIS) reference model and a set of lessons learned.\u003c/p\u003e\r\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\r\n\u003cp\u003eThis talk will describe the history of this data curation work and its culmination through the development of the JHU data archive. Additionally, the talk will feature challenges or opportunities related to data management that span different types of data from a diverse array of communities or organizations.\u003c/p\u003e"]},"provider":[{"id":"https://amiastreaming.aviaryplatform.com/aboutus","type":"Agent","label":{"en":["AMIAstreaming"]},"homepage":[{"id":"https://amiastreaming.aviaryplatform.com/","type":"Text","label":{"en":["AMIAstreaming"]},"format":"text/html"}],"logo":[{"id":"https://d9jk7wjtjpu5g.cloudfront.net/organizations/logo_images/000/000/016/original/AMIA-Logo-17.jpg?1556650205","type":"Image"}]}],"thumbnail":[{"id":"https://d9jk7wjtjpu5g.cloudfront.net/collection_resource_files/thumbnails/000/035/692/small/Screen_Shot_2019-05-01_at_9.06.03_AM.png?1556719633","type":"Image","format":"image/png"}],"items":[{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692","type":"Canvas","label":{"en":["Media File 1 of 1 - Sayeed_Choudhury_Final.mp4"]},"duration":1820.735,"width":640,"height":360,"thumbnail":[{"id":"https://d9jk7wjtjpu5g.cloudfront.net/collection_resource_files/thumbnails/000/035/692/small/Screen_Shot_2019-05-01_at_9.06.03_AM.png?1556719633","type":"Image","format":"image/png"}],"items":[{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/content/1","type":"AnnotationPage","items":[{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/content/1/annotation/1","type":"Annotation","motivation":"painting","body":{"id":"https://aviary-p-amiastreaming.s3.wasabisys.com/collection_resource_files/resource_files/000/035/692/original/Sayeed_Choudhury_Final.mp4?1556719412","type":"Video","format":"video/mp4","duration":1820.735,"width":640,"height":360},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692","metadata":[]}]}],"annotations":[{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674","type":"AnnotationPage","label":{"en":["AUTO_Sayeed_Choudhury_Final.mp4 [Transcript]"]},"items":[{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/1","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"thank you Chris thank you for the invitation here as well I have to admit I feel a little bit like %HESITATION the new kid on the first day at school I I see know very little about audio and video I I don't work with them directly %HESITATION I work with a lot of scientific research data that does include audio and video and I'll talk a little bit about the implications there are actually met Chris through a Blue Ribbon task force on sustainable digital preservation and access and that the %HESITATION final symposium for this task force %HESITATION Jon Landau gave a talk about avatar and what I found out at the time it is the Oscars the science and technology council particular are currently obligated to preserve any film that gets nominated for Best Picture ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=11.93,56.83"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/2","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and went avatar was nominated for Best Picture %HESITATION they they had a problem on their hands because this was so fundamentally different than anything they had dealt with previously it changed the landscape and I'm going to basically talk about that kind of a moment in time if you will that's happening over and over again lots of different research disciplines %HESITATION that you might find at the university and I hope find some common areas and common ground because as strange as it sounds some of the work that we're doing with scientific research data may actually have some relevance with your work and I certainly have heard things here today and and previously that have relevance for our work so I'm really hoping just to find some areas of of common interests ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=57.73,97.74"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/3","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"where I come from is the libraries at Johns Hopkins we are charged with building a large data archive of for all of the research output of our university ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=98.95,108.89"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/4","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"Hopkins is the largest recipient of federal grant funding dollars for the last thirty thirty five years I don't know exactly the right number just to give you a sense of the order of magnitude we get just under two billion dollars funding from the federal government in any given year the vast majority of that is medical all but those you might imagine there are lots of other disciplines lots of those grants so we have a fairly large %HESITATION repent or problem on our hands in even beyond the university many many if not most of our faculty members and are collaborating with our colleagues at other institutions and universities and I'll be talking a little bit about one of those projects I'm glad for the past few years something called the data Conservancy %HESITATION this was a ten million dollar war to the National Science Foundation I've been the principal investigator or their project director for that and a lot of the work that you on that I'll be talking about %HESITATION comes from that particular effort ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=109.95,163.69"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/5","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so just quickly via the sort of the points that I want to lay out for you we've been using %HESITATION what we call a data management stack model on to describe some of the work we do and all I'll go over that a little bit I want to include a definition for preservation because one of the interesting things I note about coming to conferences why don't typically attend is we may be talking about the same things but we use different words %HESITATION we may be using the same words and we talk about different things so I think it may be helpful at some at some level for me to give you this definition of what constitutes preservation and our contacts the the Sloan digital sky survey or SDS us is a large community %HESITATION digital astronomy project that has been running for over twenty years now we've been involved with the community for over a decade of the twenty years %HESITATION and that's the main a case study that'll all be describing today and then a few comments about possible similarities in terms of our communities our problems and make a purchase and then %HESITATION a call for maybe what are some grand challenges we might try to address together what you see here are is a poster that we submitted to something called the international due to curation conference a couple years ago and you don't need to worry about all all the sort of nitty gritty detail on the sites %HESITATION but what I think would be most important focus on is this center kind of stock here you see this queue ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=165.31,246.03"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/6","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and therefore rose in that %HESITATION particular stock their storage archiving preservation and curation ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=247.08,253.52"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/7","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"now one of the things that happen Hopkins when we start first started dealing with our researchers in terms of data management is they would use these words interchangeably they would say I am preserving my date I don't need your help %HESITATION or I I know what to do when I'm archiving the data and so on and what we found is that not surprisingly these are experts in their domain they're not necessarily experts in data management %HESITATION they were using these terms to mean different things and we thought it was really important to come up with a common framework and a common models so that we could have communication with them and I'm not implying this is you know it's a canonical model that everybody needs to use %HESITATION but it is one that we have found very helpful in our conversations at Hopkins and now beyond I'm told that this is being used in some of the live in information science schools and some of the data manager training that takes place within particular communities and and domains and it's a hierarchical model in the sense that storage is necessary but not sufficient for archiving and archiving store is necessary but not sufficient for preservation and so on up to curation ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=254.36,319.98"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/8","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and by storage you know I mean what you might imagine bits on tape this in the cloud %HESITATION with some sort of back up and restore capability ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=320.84,329.06"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/9","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"archiving the next layer up in the stack is where you start to talk about data integrity data protection things like fixity %HESITATION identifiers to the you can validate that the data in fact have not become corrupted either overtime or because the media issues or because of transfer issues and and so on ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=330.01,347.37"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/10","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and then the next layer is what we call preservation %HESITATION and I will give a definition a more detailed definition than the next slide but it's really mainly about the context it's about the meditator it's about the representation information that you have to attach to the data in order for it to be interpreted and to be used %HESITATION and use over time and then ultimately at the top of the slayers on what we call curation and by curation what we mean is that it is used and re used an unanticipated ways ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=348.75,380.08"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/11","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so in a research team astronomers with the Sloan digital sky survey for example create the data for a particular purpose and they created and treated and %HESITATION describe it with those purposes in mind ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=380.92,394.03"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/12","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"but increasingly the kinds of problems were looking in our society take your pick climate change %HESITATION security internet security %HESITATION you know crime it doesn't matter they're all becoming large multi disciplinary kinds of problems and you need to bring lots of different kinds of data into the mix and lots of different kinds of methods into the mix and that doesn't happen unless you have curation ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=394.87,418.99"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/13","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so the ultimate measure in successive curation is that someone who didn't create the data can come along and use that a an intense someone anticipated way one of the people that we've been working with through the data Conservancy someone in Ruth door %HESITATION she's a data science scientists at the national snow and ice data center and during one of our meetings he somewhat sort of casually made the statement about what she thinks of state preservation ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=419.93,444.01"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/14","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"this is actually consistent with the L. A. I. S. reference model for archiving preservation so if you're familiar with that model the should look a look familiar to you but as you can see it's really about somebody other than the original data produced in being able to use your data so right now in many scientific domains if I want to use data that you produced I basically have to contact you ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=444.89,468.47"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/15","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and then you have to basically agree to share your data with me and then you have to tell me how the data is described so for example if I said I am doing research in an area that you're exploring could you please send me the data you have for temperature insulin the readings in the ocean ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=469.73,486.25"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/16","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"you could say no and that would be the end of it you could say yes and you could send me a spreadsheet and while I open the spreadsheet I could look at the column headers and C. S. B. two G. R. nine and not know what any of those things mean I might be able to infer that what these readings are but if I don't have the context if I don't have in some cases code books that but on things of that nature just sending me the data is not enough ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=487.09,512.03"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/17","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so over time this becomes even more complicated ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=513.38,516.31"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/18","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"because I may want to give you my dad I may have forgotten what those column headers are I'm not deliberately trying to make somewhat challenging for use my data I just may not remember so a successful data preservation activity is one where we are no longer idiosyncratic about how we share data what we're systematic so that I don't have to keep going back to the original date of producer and hoping that somehow there if there's a way we can interface with each other ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=517.16,542.58"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/19","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so let me turn to the Sloan digital sky serving tell you a little bit about this project it started as a set of %HESITATION twenty years ago it was a large community based effort it continues to grow and it was in many ways an unprecedented project an exemplar if you will of an early will be calling %HESITATION cyber science review sites type of project ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=546.11,565.42"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/20","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"in a couple of days of the Sloan digital or is this a survey they acquired certain types of astronomy data that spanned and and and %HESITATION surpassed all of the acquisition of astronomy did %HESITATION prior to that so just to give you a sense of the scale to give you a sense of where we are today the modern astronomy projects that are coming online in the next few years in a week hello choirs much status as the assistant so there is an exponential growth in terms of how much data is being captured by these large scale projects ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=566.28,596.81"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/21","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"it's also something that in many ways democratized if you want to call it that the use of astronomy data ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=597.75,603.03"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/22","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so the astronomers with SPSS acquired it and then eventually published from a website called sky server and they do so the four in the former data releases as they call them there are two common publications ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=603.87,615.18"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/23","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and anyone can use Chrysler and anyone can run queries against guy server there are apparently about ten thousand professional strong numbers in the world and there are over a million registered users of sky server not everyone of them is active no doubt but even if a tenth of them are active that's ten times more than there are professional astronomers ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=615.8,636.91"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/24","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"astronomers have taken images from SPSS and move them into something called galaxies %HESITATION ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=637.83,642.39"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/25","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"which is a a website you can go to your register take a brief tutorial and then you start classifying galaxies you're even images of the galaxy and your **** this elliptical is a spiral is in a regular and saw ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=643.23,656.08"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/26","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"but there was a touch screen feature that was doing this particular project %HESITATION galaxies and she discovered a blue object in one of the images and they asked and she asked his strong review should fix this there's something wrong here and the astronomers looked at it and repeatedly kept saying we don't see anything wrong with this image we're not really sure what to tell you and they pointed a telescope at that portion of the sky it turned out she discovered new object and in fact has been named after her so they there's actual scientific contribution taking place now at this point to these kinds of websites this is a slide that shares with call the data flow of the SPSS data ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=656.98,694.5"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/27","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"if you start at the top left corner is the actual telescope %HESITATION New Mexico that they've used to map of the northern sky using yes yes us %HESITATION for the SPSS data ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=695.37,705.54"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/28","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and if you keep going you know downloaded to the right you go through these levels of data so the astronomers call the telescope data level zero and they're literally bits the ones and zeros that come straight off the telescope and there may be some calibration data there may be some environmental data you know temperature data saw on things like that I am told that there maybe a dozen people in the world you can interpret that data ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=706.46,730.53"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/29","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and we've always joke please don't put them in the same room please don't put them on the same plane because they these people basically the only ones who can take that data and move it down this processing pipeline when you get to something like level one it is something that can be processed by an institution so the Fermilab in Chicago is where those data are sent and then there's a team of individuals who now start to work with it and use it but you still pretty much have to be an expert in that instrument and that particular kind of data ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=731.5,759.28"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/30","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"by the time it reaches circle level to %HESITATION it's now being housed in more familiar kinda technology stuff that is you know you can buy stuff that's commodity salon and many astronomers in the world would be able to say yeah I understand what this is now I can interpret this I can use this and at the very bottom right is the level three %HESITATION data or these databases the data releases that I describe that are put up on sky server and at this point anybody can use them in I I really do mean anybody when the sky server went live there's a period of about six months where they tracked the usage of it %HESITATION and the strong numbers at Hopkins discovered that a high school in Orlando was the second largest user of sky server ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=760.12,801.55"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/31","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so I don't know what this says but they concluded they were about to be hacked so they shot off access to this particular high school and the principal of the high school contacted them and said I have kids using real scientific data doing real work and you just cut them off so to the astronomers credit they turned around and said we will not create a whole suite of K. through twelve kind of educational applications and song so these data releases are really accessible I don't just mean you have to be an expert on %HESITATION mean you have to know about astronomy %HESITATION you can use them in any way you want ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=802.44,835.05"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/32","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"when we started our work in terms of the archiving of preservation these data %HESITATION I have to admit we were over while we we sat that none of us are strong numbers my colleagues and I don't have any professional training astronomy %HESITATION we sat there and started talking to these astronomers who would use terms would use talk about not the son we knew nothing about ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=836.09,855.86"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/33","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"%HESITATION we also knew we were talking about a lot of data at the time we didn't know how much but we had a sense that this was a lot so we try to do something that was a little bit more tractable and basically said is there something we can use that's familiar in terms of our world in terms of the library in terms of publishing in terms of education and saw ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=856.74,874.88"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/34","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"what they said was you know there's actually a fourth level of data so once we take these data releases and put them out we do our analyses we derive smaller chunks of data and those get cited an academic papers so maybe you can start there ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=875.72,890.3"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/35","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so in essence that's what we did we started looking at papers in astronomy and looking at figures and tables and things of that nature and sing all those are the level for data let's try and work some sort of preservation act to be around those over time they have asked us to go back the pipeline ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=890.99,907.64"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/36","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so initially we started with a level four and they said okay we think you're getting a handle on this now can you do with the databases themselves I think we're getting hand around us about the level two and not that long ago the lead astronomer Hopkins is named Alex lay actually came to me and said there's data sitting on the telescope in New Mexico I want you to go to New Mexico and get ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=908.48,931.5"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/37","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"%HESITATION I I have no idea what the data is that what what what the value is but he seems to think that their stuff that still hasn't been collected so over time we've been pushed up this pipeline and asked to deal more and more %HESITATION with the wall on filtered in a process data ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=932.6,946.96"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/38","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so this is a description a very high level description course of what kind of data you would see in the Estes us and in a raw date of course as you might imagine it can be even the bits themselves there's two kinds of data that's been encapsulated in this are that important to note one is a catalog which basically tells you about the date outs connected what you can do with it in some sense that code book that I talked about earlier and then this is data archive server ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=949.48,975.9"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/39","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"astronomers the basically taken a lot of these data and put them into a database and then come up with new techniques and methods to access query and run analyses against those databases and they did that basically because they were overwhelmed by the scale and complexity water collecting ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=976.83,994.12"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/40","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so you probably hear a lot he read a lot about big data %HESITATION terms gets used and and the mainstream press even at this point and there's lots of definitions about big data that talk about volume velocity discuss the variety and then those are all important and useful but they're all very data centric definitions the definition that I've been using about a big data is when a community has to develop new methods it's big ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=994.96,1020.38"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/41","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"because they've been overwhelmed and they have to come up with new ways of managing describing ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1021.22,1026.27"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/42","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"eating the data and then run the methods against that's what happened to the astronomers with us yes us ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1027.15,1032.89"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/43","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so as we sat there and started thinking okay we're going to become one of the long term homes that are cause for these data what are we up against and we basically started taking some of the data releases we started moving up that pipeline in bringing data into our environment ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1033.76,1047.48"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/44","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and then we were ready at least we thought we were ready to take the whole thing it and we found out it's about a hundred and sixty terabytes of data and we do have it all on site now and we do have two copies and we don %HESITATION the archiving work but there was a little conversation earlier about ingestion processes and work flows and pipelines we were completely broken by the in just a process of bringing the Estey assisted and I'm not trying to be dramatic %HESITATION the system administrator my chief IT architect in the software developers would literally come to me and say can we coming back out of this really do we have to do this because this is really killing us and I said no we we we cannot back out of this thing to have to figure this out so I'll give you a taste of one kind of problem we encountered there's a bit of a misconception that a big data this is be huge blobs of data seven hundred sixty terabytes maybe there's no ten terabytes chunks or something ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1048.14,1106.35"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/45","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"this is strongly data there are eighteen million files ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1107.12,1111.27"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/46","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and some of them are really small on the order of kilobytes ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1112.82,1116.61"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/47","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"we did not anticipate that ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1117.56,1118.96"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/48","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and we are ingesting all of these files are stored system and it is grinding to a halt ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1119.82,1125.1"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/49","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"repeatedly and we're not sure why even the astronomers couldn't told us they're eighty million files they will tell you another a lot I mean clearly it's more than ten it's more than a hundred more than a thousand but if I said it's eighty million they would say now of course is not nearly that many files ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1126.34,1142.67"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/50","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so the storage system we're using is not configured to deal with large fast flows of eighty million files into its into its environment this was compounded by the fact that we are keeping the copies on tape on LTO four and now moving to LTO six ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1143.49,1158.05"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/51","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so couple of consequences of this first thought was okay we have the package the stuff up here we don't know quite frankly with the preservation implications might be of packaging it because we can tell you how we made the package or not astronomers so we may not be packaging it and sort of semantically meaningful ways we're just doing from an efficiency perspective ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1159.48,1181.84"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/52","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"but we have to do this the system simply can't operate properly in this way so life is good we package them up we bring them in the system performance improves patting ourselves on the back with thinking life is good ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1182.68,1194.39"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/53","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"then we started to run fixity our attack attach fixity information to ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1195.26,1199.41"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/54","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and we discovered that the fixity wasn't granular enough ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1200.91,1203.41"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/55","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"it was either happening at the block level of the storage system itself which is what you got out of the box or the package label ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1205.02,1211.58"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/56","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so what happens if you run a fixity check later on and you're not sure of the block that's been compromised if it's the package has been compromised or if it's the underlying data which elements of the underlying data amongst these were not anticipated problems we did not think about them in advance we've been sort of retro actively trying to deal with them and we've been talking to our storage vendor about this problem into multiple storage vendors about this problem and they acknowledge it ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1212.95,1238.67"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/57","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"but I'm not really sure they've given us a solution to it ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1239.6,1242.3"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/58","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and as we may be moving into object based storage the question is do you keep things in files to keep things in databases or do you start thinking of them as objects there are profound implications for preservation perspective about this particular set of questions ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1243.14,1257.71"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/59","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and I I don't have answers for you I'm just saying they're very important questions that we have for us ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1258.87,1263.49"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/60","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so for all all of the suffering that we've had over the last few years of the th we took which I thought was %HESITATION are fairly expected and predictable thing that an academic library would do which was we put it out there for all to see ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1266.42,1282.48"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/61","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"%HESITATION we basically said here the things we did they didn't work the things we try to fix that and here the things that may be what one okay ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1283.36,1291.48"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/62","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"what a surprise by is many of my colleagues who are gonna economic libras basically thank you for doing that ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1292.39,1298.34"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/63","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"nobody else is done ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1299.27,1300.4"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/64","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"I can't go to some other public place and say here the issues we dealt with in ingesting large amounts of scientific did into our repository on to our archive you seem to be the only guys going to do this I think that's a problem I I mean either the other institutions are facing up they haven't dealt with this kind of problem or they solved it which would be great and we definitely like to hear about it but we think it's really important to put this information out there and try and find other not just economic Leiber's universities but other content managers of the call that who may have similar problems so that we can start to to talk to each other and work with each other ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1301.96,1338.78"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/65","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"we classify these lessons learned according to that stack model that I should get the earlier slide so it's broken down into storage archiving preservation curation ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1339.62,1348.35"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/66","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"the agreement that we had originally with the SPSS folks was to preserve their data we haven't gotten there we were so stock on the archiving piece %HESITATION that we're only now beginning to address some of the production problems but they're still pressing problems ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1349.6,1363.19"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/67","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so a little bit about what I think of some similarities in terms of some of the issues that I raised here ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1365.87,1370.7"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/68","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"well I mentioned to you that it was actually a presentation by Jon Landau the producer avatar that started making you think about these commonalities and when I told him about this work yeah she said yeah I think there are some similarities there because when it comes to something like avatar they're producing these sort of wall data files if you will and then generating a processing them to create the things we see on the screen and there's an interesting question about which of those things you preserve do not care to just redo everything party trying to keep some of them and then use them later on in many ways James Cameron was a pioneer in terms of what he was doing with film and the way these astronomers were with their data they go ahead because there's an interesting problem interesting creative kind of activity they want to work with and they just for trying to head the build new instruments maybe new data formats certainly new methods and so on and they do something really amazing but they don't think about preservation ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1371.69,1424.49"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/69","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"the astronomers Gibson we preserve the data and we would talk to them about what do you mean by that and then they say actually we have preserved the data here you go because of the data ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1425.59,1433.62"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/70","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so it's you know to be careful what you wish for moments of it's an incredible opportunity quite frankly from where I sit it's an honor that they would trust us to do this but it it's not something they've considered in any way in an explicit manner as the start of this even in astronomy they're generating so much today and now that they can't even keep all of the data on their drives have to analyze it and then dump it off their systems so this is only going to get harder not easier and the kinds of technologies we've been using to date I don't think a particularly well ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1434.5,1467.24"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/71","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and it's not even the technologies themselves it's all for the approaches that we have sort of the processes these assumptions we have about policies %HESITATION yeah of course we have to two hobbies it's really expensive and challenging to have two copies of course we can keep it on tape it's really hard to read the packages off the tape is really our bundle I'm off the tape so even the policies we've generated need to be questioned ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1469.54,1493.46"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/72","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and I think you know I'm preaching to the choir and you know I think the FBI presentations awesome in terms of timing but this we're dealing in new formats new machines new ways of acquiring data ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1494.73,1505.28"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/73","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"as you know the last couple weeks of Baltimore been very interesting for us and a lot of that started because of one person's hand held cell phone video ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1506.18,1514.8"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/74","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"it created a ripple effect that we're still feeling in the city ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1515.64,1519.35"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/75","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"there's also the question of potential value it's difficult to know what the potential value might be I asked some astronomers suppose were using data and it isn't used for five ten fifteen twenty five thirty years can we D. exception can we move it out of the archive and in essence platoons them they're saying this used to be here but we got rid of it ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1521.18,1541.89"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/76","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and they said we know what you're asking we got it it's expensive it's hard it's almost off here's a real scenario you do that and the new twenty six years a supernova occurs in the sky and the strong numbers that that day go %HESITATION we know the song folks not that portion Scott and we know Hopkins as the date are cut the top %HESITATION you got rid of it ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1542.73,1561.49"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/77","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"how do you assess that in terms of keeping versus keeping are not keeping ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1562.79,1567.07"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/78","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so many of the industry's represented here I think have very different audiences a very different kinds of access provisions and so on but I will also say that in terms of scientists film video audio is an incredibly important part of the research they do and some films and some recordings on become part of cultural heritage so there starts to be the sort of overlap between academic with a cultural institutions maybe the private entertainment industry and so on ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1570.12,1598.27"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/79","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"there are scientists here at Hopkins who look at turbulence float so it's not even be that's generated by observing something it's simulations but they're still videos %HESITATION and not surprisingly many of them are also saying what are you going to do for us are you going to help us and I'm hoping that you have some answers I can give them ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1599.81,1617.26"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/80","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"I think the technology requirements may start to convert at least at the lower levels of that stack %HESITATION and if they do I think there's some value for us in terms of working together and talking together ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1618.59,1629.59"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/81","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"Sir the the last of the technical concept all put out there is what we call the information graphs of this is the project being funded by the Sloan foundation called arm out and there's a a protocol in RDF based particle called the object we use an exchange or O. R. E. the talks about how you build compound objects that connect publications that connect data that connects off where video audio you name it and creates these information graphs and what those graphs do in some senses give you a sense of how things are connected to each other ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1631.04,1664.29"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/82","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and what's the program notes where were things derive from what machines are people touch those that and what did they do with them ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1665.13,1671.01"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/83","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and I do know that in some cases if you're I was having lunch and talking with %HESITATION to refund all about Major League Baseball this amazing work they're doing and the question I asked was are you using the same equipment in every state and he said yes and I thought wow I wish I could do that ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1671.85,1686.22"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/84","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"I wish I could tell all the scientists please use the following technology ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1687.17,1690.49"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/85","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"I understand you have a great to do control in terms of what acquisition mode use but think about news so the news media covering Baltimore had its cameras that I'm sure industry standard and they know exactly what to do with them but people are taking video of fires people were taking pictures of things are happening that's part of the news as well ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1691.33,1710.87"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/86","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and how you include that how you preserve that is not clear to me but we're hoping that these information graphs maybe a way to start expressing all those connections expressing the proconsul that we can start to think about more explicit ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1711.86,1724.46"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/87","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"not earned all and with the slide that I I call call some grand challenges ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1726.57,1730.24"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/88","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"attention we've been talking to some of the storage vendors and they they are receptive and they're open to hearing what we're saying but I will also say there's a bit of a sometimes not even an undercover and explicit one of ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1731.7,1742.62"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/89","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"you know this is this is not a common problem what you're describing is such an edge case that if we built something just for you you know it's not marketable it's nuts on saw ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1743.6,1752.78"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/90","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"I can't help but think of the libraries and other universities are going to this as well but I also can help think that other people another indices are going through this as well ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1754.26,1762.99"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/91","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and it's not going to be enough for Hopkins or even universities to go to the storage industry and say we moved you to do these things for preservation perspective I think many industries are going to have to come together and make this kind of message %HESITATION take this kind of message to them ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1763.83,1778.47"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/92","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"and then the final thing I'll talk about is I have asked many of my colleagues and I read lots of papers show me an example of a successful format migration scientific research data and I've yet to see one if you have such examples in your community we would love to learn from you because this is a bridge we have yet to cross but I'm sure we will do so soon enough ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1779.42,1800.05"},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/93","type":"Annotation","motivation":"transcribing","body":{"type":"TextualBody","value":"so all end with the feud nonsense and a few resources you can look at for further information if we have time to be happy to quest thank you ","format":"text/plain"},"target":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692#t=1801.46,1809.02"}]},{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674","type":"AnnotationPage","label":{"en":["English [Transcript]"]},"items":[{"id":"https://amiastreaming.aviaryplatform.com/collections/48/collection_resources/4799/file/35692/transcript/3674/annotation/94","type":"Annotation","motivation":"subtitling","body":{"type":"TextualBody","value":"https://d9jk7wjtjpu5g.cloudfront.net/file_transcripts/associated_files/000/003/674/original/transcript_1558512391.vtt20190522-6031-1uylbig?1558512391","format":"text/vtt","language":"en"},"target":"https://d9jk7wjtjpu5g.cloudfront.net/file_transcripts/associated_files/000/003/674/original/transcript_1558512391.vtt20190522-6031-1uylbig?1558512391"}]}]}]}