See this is the sort of thing that drives me nuts ...

Look, I get that we (and I'm including me) are not perfect. I think that imperfections make us much more interesting and I appreciate that.

But can we PLEASE start to maintain our code/vignettes/library/tutorials or whatnot? I mean, it really burns me when I go through the trouble of reading the vignettes, reference materials and whatever tutorials have been released to the internet and ... after hours of my time ... I'm still scratching my head wondering what the BEEP.

It should also burn the creators of these documents that they have spent time making all of these supporting documents and people still cannot use their libraries.

Here's an example (and I apologize for picking on this particular library):

Chip Analysis Methylation Pipeline for Illumina HumanMethylation450 and EPIC

or ChAMP is one of two pipelines I can find that will analyze Illumina EPIC arrays in Bioconductor. I have two objectives: 1) analyze my data (obviously) and 2) compare both pipelines. Now to be fair, I'm having issues with the other pipeline (RnBeads) as well, but ChAMP has a great example how something so minor can be time consuming.

So there are two ways to approach this library:

1. You can run all modules using the function champ.process()

2. You can run each module and subsequent steps individually 

Since I am comparing libraries I am approaching things using both options. In theory both options should give you the same outcome given that you use the same arguments. When I run each module (option 2), I have to set up my directory and load my data with champ.load(). The loading function requires a sampleSheet.csv file. This file includes your sample names, Sentix ID, Sentix Postion, etc. All that good stuff that comes with analyzing Illumina arrays. No problems.

But then I get to champ.SVD() and I want to include an argument with even more sample information not contained in the SampleSheet.csv. I read the reference on the function and two things pop out:

Argument        Description

sampleSheet

If the data has not been loaded from .idat files and fromFile=TRUE then this points to the required sampleSheet. Default is "sampleSheet.txt"

studyInfoFile

If studyInfo =T, this file will include the additional study information. Default is "studyInfo.txt". studyInfoFile If studyInfo =T, this file will include the additional study information. Default is "studyInfo.txt".

Whoa ... wait a minute. Is the sampleSheet suppose to be a CSV or a TXT. To some people this may appear to be a small issue, but for me this is sort of big. CSV files are separated with a comma between fields (columns) whereas TXT files are separated with tabs. This really matters because data loads use parsers to tell the computer where each field can be found based on what separates fields (a comma or a tab). File format can seriously screw up a pipeline (we've all been there). 

So my thought process goes like this:

1. This library has been updated but no one updated the reference manual .. so all files now have to be CSV. So I create a studyInfo.csv and that fails. sampleInfo has to be a TXT file.

2. This library has been updated and sampleSheet now has to be a CSV file to reflect a new change but studyInfo still needs to be a TXT. Well that runs, but it doesn't seem to include anything form the studyInfo file. So .. that fails ... sort of.

3. This library has been updated but champ.SVD() has not been changed to reflect the changes to champ.load() (which now requires a sampleSheet.csv). Well this would be super dumb because it would require me to create two sampleSheets (both a CSV and a TXT). And this also fails.

So in the end, I spent 30 minutes changing file formats and I still didn't get the result I want. To be frank, I shouldn't have to spend any time tinkering with file formats because the reference manuals don't reflect the requirements for arguments and this is a bad practice for me to do since I am just enabling the problem. The problem being that we tinker and force code instead of making authors fix their code/vignette/reference manuals. This just means that the next person will have to face the same problem I just walked through. 

I would like to see libraries (including vignettes and reference manuals) updated and libraries that do not get updated as errors are reported get pulled down and become obsolete if authors are no longer maintaining code and these supporting documents.

But there is a silver lining ... these issues make me much more diligent about documenting my own code and pipelines. I now emphasize paying closer attending to details during my own documenting process.

Using Shiny Apps to Anticipate Questions from Your Audience

I like to think one of my strengths in what I do is presenting data in an interesting manner that tells a story AND anticipating questions that may arise from the presentation of data. When I was a graduate student I had an adviser who encouraged his students to add slides following a presentation for questions that we would anticipate from our audience. You couldn't cover all of the material in a 20 minute talk, but you could use the 10 minutes of questions/follow-up to add material if you anticipated those questions ahead.

Today, I apply the same method to data reports and data presentations. I cannot point out and answer all questions that may arise from data discovery in a data report, but I can use my data presentation to anticipate those questions.

Here's an example:

I have some data looking at stress levels, energy levels, etc. from six different locations. This data set will likely add more locations as it grows. The story that I want the data to show will be the data trends in each location. At the moment, I'm really not sure the importance of each location and I'm going to predict that importance of locations will change as I continue doing data discovery on this set.

Below is an example of a figure that I would add to a data report as a part of a dynamic document (see note below). With six different locations, the figure is kind of busy (and I will probably add more locations in the future)! To follow a location, you really need to outline the location and study it on its own. So I could split this figure into six figures with one location per figure. That's one solution, but if I want to compare it to the other locations I'm left having to deal with flipping between figures. Flipping figures gets messy and I know I have trouble keeping everything straight in my head.

So my solution is to include the above figure in the data report and to create a data dashboard with Shiny where I can highlight selected locations that I can then present later. This allows me to highlight all of the data from the second location to emphasize a point or answer a specific question concerning a specific location with a strong visual.

A plus to creating a Shiny dashboard for use in a presentation is that it can be given to your stakeholders to interact with at a later date. You can host Shiny apps through Shiny itself or create your own Shiny server. Hosting apps through Shiny usually makes them public and is limited to the number of apps you can host (unless you pay to use Shiny's cloud services). If you have your own hardware or use another cloud service, you can create your own Shiny server and host your apps from there. I choose the latter because I enjoy having admin rights.

Note: Dynamic documents are incredibly important for all data, but are especially important to data that will grow. I can create a single word document/HTML/presentation that will change all of my figures as new data is integrated (plus add my own unseen notes <- everybody should make dynamic documents).


SNID BITS of CODE:

I use the following function to create figures with shared legends. I found this off of stackoverflow:

grid_arrange_shared_legend <- function(...) {
    plots <- list(...)
    g <- ggplotGrob(plots[[1]] + theme(legend.position="bottom"))$grobs
    legend <- g[[which(sapply(g, function(x) x$name) == "guide-box")]]
    lheight <- sum(legend$height)
    grid.arrange(
        do.call(arrangeGrob, lapply(plots, function(x)
            x + theme(legend.position="none"))),
        legend,
        ncol = 1,
        heights = unit.c(unit(1, "npc") - lheight, lheight))
}

For my Shiny App, the UI is pretty straight forward:

ui <- shinyUI(fluidPage(
 
   # title
   titlePanel(""),
 
   # Conditional sidebars
   sidebarLayout(
     sidebarPanel(
      conditionalPanel(condition="input.conditionedPanels==1",
        radioButtons("", label = h3(""),
                     choices = list("" = 1, "" = 2,
                                    "" = 3, "" = 4),selected = 1)
      ),
      conditionalPanel(condition="input.conditionedPanels==2",
        radioButtons("",label=h3(""),
                     choices = list("" = 1, "" = 2, "" = 3), selected=1)
        ),
      conditionalPanel(condition="input.conditionedPanels==3",
                       radioButtons("Wellness",label=h3(""),
                                    choices=list(""=1,""=2,""=3,
                                                 ""=4,""=5,""=6), selected=1)),
      conditionalPanel(condition="input.conditionedPanels==4",
                       radioButtons("",label=h3(""),
                                    choices=list(""=1,""=2,""=3,
                                                 ""=4,""=5)))
      ),
   
      # Main panel separated into tabbed conditional panels
      mainPanel(
        tabsetPanel(type = "tabs",
         tabPanel("", value =1, plotOutput("")),
         tabPanel("", value=2, plotOutput("")),
         tabPanel("Wellness", value=3, plotOutput("WellnessPlot")),
         tabPanel("", value=4, plotOutput("")),
         id="conditionedPanels"
        )
      )
   )
))

Server (Redacted):

server <- shinyServer(function(input, output) {
  # Calling requirements
  require(dplyr)
  require(gridExtra)
  require(grid)
  require(ggplot2)

  grid_arrange_shared_legend <- function(...) {
   ...
  }

  # data calls

  wellness<-read.table("../wellness.txt", header=TRUE)

   output$WellnessPlot <- renderPlot({
   
     p1 <- ggplot(data= wellness %>% group_by(Test) %>% filter(Test=="Stress_level"), aes(x=Date, y=Ave_score, group=, colour=)) +
       geom_line() +
       geom_point() +
       theme_bw() +
       labs(y="Stress Level", x="Date") +
       theme(legend.text=element_text(face="bold", size=12)) +
       theme(axis.title = element_text(face="bold", size=12)) +
       theme(axis.text.x = element_text(face="bold", size=10)) +
       theme(axis.text.y = element_text(face="bold", size=10))
   
     p2 <- ggplot(data= wellness %>% group_by(Test) %>% filter(Test=="Perceived_health"), aes(x=Date, y=Ave_score, group=, colour=)) +
       geom_line() +
       geom_point() +
       theme_bw() +
       labs(y="Perceived Health", x="Date") +
       theme(legend.text=element_text(face="bold", size=12)) +
       theme(axis.title = element_text(face="bold", size=12)) +
       theme(axis.text.x = element_text(face="bold", size=10)) +
       theme(axis.text.y = element_text(face="bold", size=10))
   
     p3 <- ggplot(data= wellness %>% group_by(Test) %>% filter(Test=="Energy_level"), aes(x=Date, y=Ave_score, group=, colour=)) +
       geom_line() +
       geom_point() +
       theme_bw() +
       labs(y="Energy Level", x="Date") +
       theme(legend.text=element_text(face="bold", size=12)) +
       theme(axis.title = element_text(face="bold", size=12)) +
       theme(axis.text.x = element_text(face="bold", size=10)) +
       theme(axis.text.y = element_text(face="bold", size=10))
   
     p4 <- ggplot(data= wellness %>% group_by(Test) %>% filter(Test=="SatisfiedwHealth"), aes(x=Date, y=Ave_score, group=, colour=)) +
       geom_line() +
       geom_point() +
       theme_bw() +
       labs(y="% Satisfied w/ health", x="Date") +
       theme(legend.text=element_text(face="bold", size=12)) +
       theme(axis.title = element_text(face="bold", size=12)) +
       theme(axis.text.x = element_text(face="bold", size=10)) +
       theme(axis.text.y = element_text(face="bold", size=10))
   
   if(input$Wellness == 1) {
     p1 <- p1 + geom_line(data=wellness %>% group_by(Test) %>% filter(Test=="Stress_level" & Location==""),size =3)

     p2 <- p2 + geom_line(data=wellness %>% group_by(Test) %>% filter(Test=="Perceived_health" & Location==""),  size =3)

     p3 <- p3 + geom_line(data=wellness %>% group_by(Test) %>% filter(Test=="Energy_level" & Location==""), size =3)

     p4 <- p4 + geom_line(data=wellness %>% group_by(Test) %>% filter(Test=="SatisfiedwHealth" & Location==""),size =3)
     grid_arrange_shared_legend(p1,p2,p3,p4)
   } else if (input$Wellness == 2) {
     ...
   } else if (input$Wellness == 3) {
    ...
   } else if( input$Wellness == 4) {
     ...
   } else if (input$Wellness == 5) {
    ...
   } else {
     ...
   }
   })
})

Good News for the Market

The great news for the market is that Bioinformatics is going to continue to grow with a CAGR (compound annual growth rate) approximately 20.4% by year 2020. Bad news is that bioinformatics market is hindered by a "lack of inoperability among data formats" and a lack of talented, skilled professionals.

So lets first address the "lack of inoperability among data formats". In other words ... as a profession we lack standardization. We write code to fit the test data we are using and the test data might not be in the same format as your data. So this leaves you having to either reformat the data or re-write the code. And this cycle repeats itself. If you have been following along with my posts already, you know that lack of standards drives me (and others) nuts! This is especially infuriating for researchers who are mostly interested in applied bioinformatics and half of the reason why almost every science lab needs a bioinformatician (the other half of the reason is that "gone are the days of Excel"). Two things are going to drive this change. First, clinical research and big pharm cannot afford to allow this nonsense to go on. This is why there is a big boom for commercial bioinformatics software designed to run and re-run the same analyses over and over again (a production pipeline vs. a R&D pipeline). I believe that those of us writing bioinformatics code hoping to one day be commercial will start writing in a standard that will be more appealing to buyers and thus creating a standard. Second, I really think there will eventually be a little group of bioinformatics professionals who will release guidelines to hold the profession accountable for. The only reason I can think as to why this hasn't happened is that the market is growing too rapidly for anyone to take a moment to approach this.

Now to address the lack of skilled and trained professionals. For all purposes, bioinformatics is a data science field and it competes for data scientists with other industries. As I mentioned in a previous post, there are colleagues of mine that I have personally known to have been trained in bioinformatics who find employment in other fields. As depressing as it is, when you Google "bioinformatics as a profession" you will stumble on at least one rant from someone leaving the profession for another career. It is true that the market is growing, but so is the market for data scientists in general. Bioinformatics programs combine data science training with training in biology (you have to know something about the biology). Glassdoor estimates that the national average salary for a bioinformatics scientist is $85,149. The national average salary for a data scientist is $118,709. Keep in mind: bioinformatics is still tied strongly to academia (where salaries in general are much lower) and there are certain biotech hubs where salaries for bioinformaticians are much more comparable to a data scientist. Every business is looking for a data scientist and the field of data science has also had to define itself and is still defining itself (here is a really fun discussion on YouTube by the Royal Statistical Society). Not only are salaries better, but these jobs are not always restricted to a few hubs. Plus there is less of an emphasis on degree and publication record for other data science positions outside of bioinformatics.

So how can we make bioinformatics more attractive?

1. Pay better. Although this is much harder to achieve given bioinformatics is still deeply rooted in academia and sadly biologist already lose out in earnings of PhDs (the group of people most often in need of applied bioinformatics).

2. Replace people with tools. R-bloggers approached this back in 2012 with data science and I would agree with this opinion. Bioinformatics isn't just technical, but sociological. There will always be a need for people who understand the principles and methods of bioinformatics to adequately communicate about the data, its findings and to advocate for future approaches.

It will be interesting to see how the market will look in the next 5 years and some real data on graduates after earning degrees in bioinformatics. I would like to think that as companies place emphasis on data scientists ability to save millions of dollars, the bioinformatics industry will place emphasis on bioinformaticians ability to improve quality of life. Or, more likely, health insurance companies placing emphasis on bioinformaticians saving them money with pre-diagnostics... 

Blog Description - I think? At least for now? Like most of my life, it is an ever evolving pipeline.

I feel the need to sort of "define" this blog. Everyone seems to have a blog today. What makes this blog different from any other? I probably should have started with this post, but, despite my overwhelming need to plan everything out first, I still have an affinity to jump into things and then figure it out later. So ...

This blog will probably never be incredibly technical. I will talk about pipelines, software, general cool tools I get excited about, etc., but you probably won't see any code of mine up here. I feel as though there are plenty of resources already available and, if I do something technical, I'll post it as a vimeo (I have a few in the making, but time is one resource you cannot replenish and you always need more of). But I do see this blog developing into a resource for soft skills. And soft skills can be equally as important as the technical skills for career development! Now don't freak out if you are slightly socially inept (or very inept). There are plenty of careers for those types of people too. But in order to explain your work to an audience, soft skills are important. And a certain amount of soft skills are necessary for a general working environment with other people (specifically gives a knowing look to network guys).

One thing that I will share on this blog are some bits and pieces of conversations with my husband (Ace). He is a bit of an economics/statistics wizard! Seriously. And a lot of what we both do at our jobs overlaps. Data mining, analytics, database and reporting are comparable no matter what the data are to start with. In fact, it isn't uncommon to find someone with training in bioinformatics in another field such as banking. Why? Well to be blunt, these individuals have the same skill sets businesses are looking for, they make more money and there are more opportunities for career growth. Personally, I am passionate about the science. I get a high from working in biotech because I see it as much needed service for human advancement. Ace gets a high from saving companies millions of dollars. I get a high from helping with the research and development of biologics that will save millions of lives. To each their own.

But since Ace and I have similar skills and have different areas of expertise, we often look at data from different angles. This is super cool because we can pick each others brains, but the trick is that we have to explain our data in terms the other can understand. This is a great exercise for each of us because it helps us work on explaining complex concepts without all of the jargon. This can be fairly difficult for me because it also requires an explanation of the science behind the data and that adds to the complexity.

It really doesn't matter what job you have, at some point you will need to be able to adequately explain your work either to your boss, a client, etc. Given different backgrounds and expertise, this can be challenging. Data scientist, IT, bioinformaticians, etc. are usually considered black boxes. A black box means that other people at work have no idea what you do, but they know it is important. This can be a good thing because it usually means no one will ask you what you are doing and you sort of get a "free" pass with some things (meaning you are generally left alone to do your job which requires that ability to "tinker" with your computer all day). BUT a black box also comes with frustration. No one understand why you don't have results yet, why it is taking so long to fix a network issues, or why you cannot just use the same pipeline someone else published with. So, I try not to be a black box and explaining things without the jargon to Ace during dinner helps me hone those soft skills necessary to help explain myself to other people I work with.

This shirt was a vendor handout at the 2015 ACTG conference and plays on how bioinformatics is a black box. I challenge my soft skills to match my technical skills so that this shirt becomes just a joke and not general perception of my profession.

Another Paper on Reproducible Research

Here is yet another paper on our lack of reproducibility. OK, we hopefully now "get it" and understand that we need to do a better job of reporting and publishing our protocols, including bioinformatics pipelines. Can we please start publishing guidelines and standards to adhere by? And then making journals enforce that those practices get followed?

So today I'm going to give three suggestions on how to better capture your pipeline using my own experiences.

Cartoon by Sidney Harris (The New Yorker)

1. Flow Charts
I am often expressing my love and appreciation for flow charts. I love visuals. I love making lists. And I super love making a list into a visual - TaDa - flow chart. Look, it just helps me stay on track and clients appreciate having something to look at while I verbally run them through the pipeline I am using with their data. Creating a flow chart can take some time (depending on how OCD you are and how much pride in your work you have prior to giving something to a client), but I truly believe that the impact to your work and your client relationship is worth the time you put into it.

2. Version Control
Oh yes, version control is all the rave. Look, it is no secret that changes in versions can give different results. Version releases are important. Releases clean up mistakes, decrease compute time, help maintain relevancy as technology changes, etc. But even a small release where rounding a number can result in huge changes in outcome. So write down your versions and MAKE NOTES when you upgrade them! This is incredibly important for someone to replicate your results.

3. DeBugging
(scene set): You have identified a new, amazing bioinformatics tool. People are blowing up twitter about it, Bioinformatics has published on it, etc. And you have a light bulb moment: Ah .. ha maybe I can use this tool for my research since the data are some what similar to what other people are using this tool for! So you start out making a new FLOW CHART to include this new tool. Should be easy, but all of a sudden you keep getting errors that you can't figure out (so you post on Reddit). You go to the documentation, nothing helps. You search all of the forums, but you either hit a dead end or find out that everyone else is having issues. You email the tools contact only to find out that they are now hired by SAS and just don't have the time to help you.


You are now frustrated (probably because someone has you on a timeline or wrongly thinks you aren't working since there are "no results") and you just know if you could get this tool to work that the result will be INCREDIBLE (I don't know why, but I always think IT WILL BE INCREDIBLE). So you start debugging by trying everything you can possibly think would be wrong. This is where we generally shut down on documenting this process. If you are like me, you get some sort of sick kick out of debugging that is OBSESSIVE IN NATURE. You can't stop to write down what you just tried because your mind is already racing to the next trial. AND when you FINALLY get that thing to work (because you probably won't stop until you get it to break to your every whim) you have the BEST HIGH EVER. And if you are anything like me, you'll start high fiving yourself, anyone who is around you ... and you will immediately forget the details of the last several hours. BUMMER. But you don't care because you are still riding that high.


I know this about myself. I accept it. But I still need to document that process so that I don't have to repeat it (because several months later I might have to). PRO TIP: SCREEN RECORD THE PROCESS. I also like to include audio because it makes the videos more fun for me to watch later. Seriously, get some friends over later, have some drinks and put that video on. If you have geek friends or have an amusing narrative style, it will be the highlight of the party.

Command Line - for Science

My morning routine usually includes my ONE cup of coffee (I'm incredibly proud of the fact I can get through an entire event filled day with just ONE cup of coffee. This is a huge decrease from where I started!) and a look through of the news. Today I went out of the norm and headed over to Twitter to start the day. I typically do not like Twitter in the morning because the majority of the Twitter activity happens in the afternoon. (See this great brief on how to post to social media!) However today was different and I'm glad I started with Twitter because I may have missed this tweet that took me to a kickstarter project for Learning the Command Line - for Science.

I posted some tips for novice Bioinformaticians a month ago and I mentioned how important it was to learn command line tools. There is a lot of emphasis on learning scripting (Perl) and/or programming (Java) languages and tons of resources available to help you. I think that is great! I also think it is great to learn command line ... and I mean learning more commands than how to change directories or create/remove directories. If your goal is to use bioinformatics tools and you are not interested in creating them, I would focus on learning Bash, a command line language such as AWK or SED and R. Other bioinformatics tools are already available to you and the three above suggestions will help you execute pretty much every pipeline available.

Bash:
Bash scripting is your default shell for both Linux and Mac. When you call 'cd' to change a directory, you are using Bash. There are LOTS of sweet commands that make Bash really useful for bioinformatics and I encourage you to check out what tutorials are already available to learn more.

Some of my favorite commands are 'find' and the loop commands such as 'for', 'while' and 'until'. 'Find' is great for finding files (genius, I know). This command is really helpful for servers with multiple users (because people tend to move files around and then forget where they have put them) and for doing things with all files in a directory and its child directories (such as moving all of your raw reads scattered in a project folder to another volume). Loop commands are really nice for bioinformatics tools that do not accept multiple inputs (mostly newer tools that haven't been through version changes). Instead of running the command to use a tool over and over again for each of your input files (ex. if you have 2+ samples that you want to process), you can just create a Bash loop once to call the command and set it up to give a new input each iteration.

Command Line Language:
I use AWK all of the time. Most of the files that are output for bioinformatics tools or files that you would download from a public depository are in a spreadsheet-like format - meaning they have rows and columns.  AWK is a great manipulator for data in this format and it is faster than writing a Perl script to input a file and read each line to blahblahblah. You get my point.

Now I feel the need to address 'manipulator for data' since out of context this phrase could sound like I am changing data. I am absolutely not changing any data, but instead maybe changing the column order or data type. For example: A chromosome column may have chromosome denoted as either 'chr1' or '1'. They both mean chromosome 1, but their data types are different since 'chr1' is a string and '1' is an integer. Data tools hate when you mix data types which is why data type sometimes needs to be manipulated so that every time a chromosome is denoted it is always denoted in the same way (data type).

Bioinformatics pipelines change so often as new tools are created that output from one tool may not be formatted for input into another tool (i.e. output for column 1 should be in column 4 as input for the next tool). With AWK you can fix all of these things with a quick script and it has a fast execution.

SED is fairly comparable to AWK, I just happen to use AWK most of the time.

R - or (said with a pirate accent) AARRRR
R is an incredible environment for executing statistics and creating really nice graphics and visuals with your data. PLUS, there has been a real effort to make its use REPRODUCIBLE (insert angels singing). I'm just going to stop there because there are lots of dedicated R users who will tell of the glories of R. Just know it is definitely worth your while to learn!

So to summarize, if you want to just use bioinformatics tools you will benefit heavily from command, a command line language such as AWK and a great statistical environment with graphics capabilities like R. Every other step in a pipeline will be filled with a tool which is pretty much plug and play with different parameters (or arguments) that you should become fairly familiar with.



Disclaimer: The above post makes bioinformatics sound "easy". I am sorry to say that it probably won't be "easy". Any decent data scientist will tell you that data anything just isn't as easy as we tend to describe it. Data are messy and data tools can be a mess too. Cleaning up messes is really frustrating. So if you feel like you are wrangling cats, then you are probably doing it right.

Reproducibility is Key - suggestions for WGS from ICGC

Nature Communications published an article from the International Cancer Genome Consortium (ICGC) concerning characterizing somatic variant calling. The gist of the article concerns the issues of reproducibility. You probably don't need me to say this, but ...

REPRODUCIBILITY IS KEY.

Reproducibility is key to data science in any field (banks would be extremely upset if your pipeline couldn't return the same results with the same set of data). And reproducibility is a key element to the scientific method.

Yet, we have a little problem with reproducibility at the moment in the field. First, our pipelines are all different.

"Calling mutations with different pipelines on differently prepared sequence read sets resulted in a low level of consensus."

Different pipelines will give you different results. It surprises me that we have to publish studies to show that this is an issue. I mean, bioinformatics tools are being written and distributed because they are different from previous tools. So it sort of makes perfect sense your results would be different. Different aligners will give you different alignments. Different variant callers may call different variants if they use different statistical approaches and are also dependent on their input.

Second, the way we sequence to generate our data is not consistent.

"Using a standard pipeline had the potential of improving on this but still suffered from inadequate controls for library preparation and sequencing artefacts."

Well... Even if we standardize the bioinformatics pipeline, variation in the sequencing center will still diminish reproducibility (If you have a good sequencing center, I recommend you stay with that center).

So, what can we do? ICGC gives their recommendations for whole genome sequencing studies:

• PCR-free library preparation
• Tumour coverage >100 ×
• Control coverage close to tumour coverage (±10%)
• Reference genome hs37d5 (with decoy sequences) or GRCh38 (untested)
• Optimize aligner/variant caller combination
• Combine several mutation callers
• Allow mutations in or near repeats (regions of the genome likely to be more prone to mutation)
• Filter by mapping quality, strand bias, positional bias, presence of soft-clipping to minimize mapping artefacts

AND

They suggest calibrating bioinformatics tools to known results and developing additional benchmarks.

Some key points I absolutely love: 1) Designing a better experiment is important to any experiment. Making sure that you have your controls and that your sequencing center is being transparent with their standards means you can have confidence in your data going into your pipelines. Plus, there is this thing about statistics that really puts the pressure on you to make sure you have the right number and types of replicates. 2) I am also in favor of optimizing aligner/variant caller combinations. I would love to see efforts made on optimizing pre-existing tools over the development of new tools (I really don't see how people can keep up with all of the new tools). 3) Benchmarking sounds really great!

What we still don't discuss is how do we set "gold standards"? Now I realize that standards will change depending on the experiment and the model system, but who will develop each standard and who will maintain those standards? Several entities have already developed standards. I happen to like The ENCODE Consortium's guidelines for NGS experiments and recommend these guidelines to PIs when consulting prior to experimentation. The American College of Medical Genetics published their own standards for variant calling this May that I also like to use. But is letting each field establish their own sort of "gold standards" the best way to approach this issue? I'm not entirely sure what the right answer is. I do think there is a need for some entity in the field to publish and maintain a guideline. Some people may argue that we do this already through previous publications in the field. I'm sorry, but I just don't see how designing an experiment because "everyone else is doing it that way" is the best approach to setting guidelines. And the work isn't just about creating guidelines, Bioinformaticians need to be aware of these guidelines prior to analysis and they need to make it a priority.

We also do not discuss how important Bioinformaticians are to these studies. This paper demonstrates that the BIOINFORMATICS WORK REQUIRES A LOT MORE THAN PLUG AND PLAY PIPELINES.  Filtering, optimizing and benchmarking will take expertise. Reproducibility will also require Bioinformaticians to be committed version control and to publishing adequate methods. This comes down to good training and standards Bioinformaticians need to commit to their profession. I hope to see publications like this make a serious impact on both future science and bioinformatics as a profession.

FYI, if you don't want to read the nitty-gritty of the Nature Communications publication, Genomeweb did a good job summarizing the paper. Enjoy!

Bioinformatics, what is that? (Everyone starts here)

Oh boy, describing what I do for a living is one of my favorite parts of an introductory conversation. And pretty much every day I have to tell someone else what I do for work now that my husband and I have moved to Dallas, TX (I could count the number of people I knew in Dallas prior to moving on one hand). This whole exchange use to cause me a little bit of anxiety. Especially if my husband were around. A conversation would go:

Me: I'm a bioinformatics scientist and I process and analyze biological data using computers.
Newbie: What?
Husband (butting in): She makes woolly mammoths and turns chickens into dinosaurs. No, I am serious! There are these crazy scientists who want to make a real Jurassic Park.

*It is really hard to backtrack at this point to not make yourself look like a mad scientist. So you have two options: 1) change the subject or 2) just go with it and proceed to tell the Newbie how you are also working on making elephants miniature so that you sell them as pets.

So I don't actually like going around and making up (but sort of true) stories about my job. It makes for some great bar conversations, but at some point people want to know what you actually do with your life (my grandparents in particular). I am going to share what my go-to response is for someone like my 86 year old grandpappy (keep in mind that my pappy quit school at 9th grade - he went on to start his own successful business and continues to work to this day *my hero*):

_____________
I am a bioinformatics scientist and I process and analyze biological data using computers. Whenever you go to build something, you usually follow some sort of a blueprint or design. DNA is the blueprint or design for all living things. If you are building a house, the blueprint directs where the building materials, such as wood and cement, are suppose to go and how they are suppose to be arranged. Amino acids/proteins are the building materials for living things. If you are building a person, the DNA for that person would direct where the amino acids/proteins need to go and in what order so that a person is the end product. There are different blueprints for different houses in your neighborhood. Every person has slightly different DNA similar to all of the blueprints in your neighborhood. A blueprint for a bridge is very different from a blueprint for a house. Same thing applies for DNA for a plant and DNA for a person. Everything is still written on blue and white paper, but the directions are different.

Small changes in the blueprint can sometimes result in barely noticeable changes in the look of the house such as moving a window over by an inch. But sometimes small changes result in big changes such as moving a grade beam a few inches so it no longer supports a wall in your house. Changes like this could make the wall immediately fall down or could gradually deteriorate the structure of your house with time. This is similar to how some diseases occur, small changes to the DNA can have very little impact on a person or could cause disease. So how do you know if a small change will have a little or big impact? If we were still talking about a house, your builder would look over the blueprint with some prior knowledge of how houses should be built and find the error. DNA blueprints are much, much bigger. Imagine trying to find a change like this in all of the blueprints of houses in Dallas. There are over 450,000 houses in Dallas. This would take the builder a REALLY LONG TIME. And it is possible the builder may even get exhausted looking at all of those blueprints that they may miss a small change of a single grade beam that is off by a few inches. We humans are not very good at repetitive analyses like that, but computers are GREAT for things like this.

So I use my computer savvy to repeatedly look at all of the blueprints for houses in Dallas to find a small change. Or rather to look at a person's blueprint to find small changes in their DNA. I can of course do this for many, many people and find patterns that are similar to certain groups of people such as those afflicted with a certain disease.
_____________

This usually satisfies people who are really interested in what I do for a living. You should of course continuously engage with the person to quickly wrap up the conversation the moment they glaze over if you determine a recovery cannot be attained. Also, be prepared to remind the person that you are not a medical doctor and please refer them to their primary care physician the moment they start to describe their ailments. I study disease, but I do not diagnosis diseases.

It is both cliche and appropriate that my first post would be this post.

Advice for Young Bioinformaticians

If you browse the Internet you will find a lot of information concerning advice or good practices for bioinformaticians. Yet, I still see students struggle who either are ignorant to all of this FREE advice or who just like to make life hard for themselves (some people live for the struggle). I am going to assume these students are not masochist and are instead just ignorant to the good advice of their colleagues. So, here is another list of good practices that I find are most valuable to my everyday life.

1. No One Likes A Quitter

Don't quit because you are frustrated. In any field managing and analyzing data is frustrating. Data are like messy little toddlers that you have to wrangle with. Just like every parent goes through it, so do all bioinformaticians. It is just a fact of life. Go take a break (or go break something) and then come back and handle your situation. It is OK to change your approach. If something does not work, go find something that will get the job done.

2. Take Notes

Take the time to write down notes. Write some notes on your installs, on your pipelines... everything. I am 100% certain that you will have to repeat the same thing again and you will want those darn notes so that you don't have to beat your head against a wall for the second time.

3. Google, Search, Ask

You spend your life in front of a computer and your most valuable resource is your browser. There are 3 steps you should always do in this particular order until you find an answer: 1) Google it, 2) search a forum for it (Seqanswers, Biostar, etc.) and 3) ask the Internet for the answers you seek! And don't be ashamed that you are on Google searching for something. Here is a news flash, we all search for things. Our field is ever growing, ever changing and no one can be all knowledgeable where they never Google for an answer to help them out.

4. Organize

Organization will save you time. I am a huge lover of flowcharts. I have flowcharts hanging up on my wall because I love them so much. Before I do anything, I make two flowcharts. My first flowchart is going to include my pipeline (this includes testing/checks to make sure things are running correctly and reporting). My second flowchart includes how I plan on structuring my directory. NEVER put all of your files in a single directory dump. Learn to create child directories and store things in a neat sort of manner.

5. Hold Yourself to Standards

Be consistent. One of the biggest complaints about the field is the lack of standardization and consistency. You will one day be incredibly frustrated by this too ... and then you will look back at your own work and realize you are just as bad about your own standards. Develop your standards and base them on what the majority of your colleagues are doing in the field. Then stick to it!

6. Learn About Your Tools

Learn something about the tools you are using. Try to learn enough about your tool to answer the following: 1) Why am I using this tool over another? (and "because everyone else is publishing with this tool" should not be your only answer) and 2) If something breaks or if I have data that are not structured as the correct input, can I debug this issue? Also, if you are using proprietary software (nothing wrong with that), please learn about its toolkit. A lot of the tools used by these software are built from the freeware versions and repackaged in a much more user-friendly GUI. So if you plan on using 2+ softwares to validate each other, you may be stacking your deck unfairly if you do not know what freeware versions your proprietary software is built from.

7. Please, No More Redundant Tools

Do not reinvent the wheel. If there is a tool out there that you can use, take the time to learn how to use it instead of writing another one. We really don't need to be overwhelmed with tools. Seriously, how many alignment algorithms do we need? 10? 50? 200?

8. Be Relevant

Be relevant by continuing your education. You have chosen a field where the technology changes quickly. You too should be able to adapt to those changes and stay relevant.

9. Take Time to Think About How to Communicate

Most likely you are reporting your analyses and results back to someone else. Take a bit of time to think about how to best explain and describe what you are doing. And remember who your audience is so that you can communicate to their understanding.

10. Keep It Fun

Remember, your job is fun! I strongly believe that the most successful careers are the ones that you ENJOY and I hope you got into this field because you like data, computers and biology. There is no shame in taking some time to give yourself an air high-five.

Your BLAST completed after a week of anticipation - high five!
You changed all 'chr1' to '1' before making that rookie mistake (it gets us all) - high five! 
You gave a great explanation of mapping (my husband thinks all I do is map things) - high five!
You just made the most complicated looking data into a great visual - high stinking five! 

Relish in all the victories no matter how small.