Somando colunas com base em critérios

Oct 10 2020

Eu tenho um dataframe que consiste em três colunas: x, ID e date_time. A coluna “x” é um registro de uma variável x, ID indica o que está sendo registrado, enquanto date_time indica quando. Veja um pedaço do dataframe abaixo.

A partir deste dataframe, gostaria de calcular um novo dataframe com sete colunas: "Medida", "ID" e "Data", “x_4_10_day”, “Day_total”, “x_4_10_night”, “Night_total”.

"Medição". Esta coluna deve informar qual número de medição era para um determinado ID. Uma medição começa às 23:00:00 e vai até às 22:59:59 do dia seguinte. Uma medição, entretanto, começa em momentos aleatórios e a duração da primeira medição não é, portanto, de 24 horas. Nem é a última medição 24 horas.
"EU IRIA". Indique o ID de uma determinada medição
"Encontro". Esta coluna deve mostrar a data da última gravação em uma determinada medição neste formato: aaaa.mm.dd.
“X_4_10_day”: uma medição é dividida em um dia (7: 00: 00-22: 59: 59) e uma noite (23: 00: 00-6: 59: 59). Esta coluna deve indicar a quantidade total de tempo (em minutos) que x esteve entre 4-10 (ambos incluídos) a cada dia em uma determinada medição. Uma gravação de x entre 4-10 pode ser considerada como x estando entre 4-10 por 5 minutos, pois há 5 minutos entre cada gravação.
“Day_total”: esta coluna deve indicar a quantidade total de tempo (em minutos) x foi medido em um dia. Existem valores ausentes em x que devem ser subtraídos. Valores ausentes de x são deixados em branco. Para cada medição em falta, 5 minutos devem ser subtraídos do tempo total. Além disso, algumas medições começaram depois das 7:00.
“X_4_10_night”: Esta coluna deve indicar a quantidade total de tempo (em minutos) que x esteve entre 4-10 (ambos incluídos) a cada noite em uma determinada medição.
“Night_total”: Esta coluna deve indicar a quantidade total de tempo (em minutos) x foi medido em uma noite. Existem valores ausentes em x que devem ser subtraídos. Valores ausentes de x são deixados em branco. Para cada medição em falta, 5 minutos devem ser subtraídos do tempo total.

Deve haver uma linha para cada medição exclusiva. Até agora tenho um código que retorna as colunas: "Medida", "ID" e "Data" corretamente:

df1$mydate = as.Date(df1$date_time, format = "%Y.%m.%d %H:%M:%S")
df1$tm <- as.numeric(df1$date_time)
df1$dts <- 86400*as.numeric(df1$mydate)
df2 <- df1 %>% 
group_by(ID,mydate) %>% 
transform(date = case_when(((dts-3600)<tm & tm<(dts+82800)) ~paste0(mydate), ((dts+82800)<=tm) ~paste0(mydate+1) )) %>% 
select(ID,date) %>%   
unique() %>% 
group_by(ID) %>% 
mutate(measurement = row_number())

porém não sei como fazer os últimos.

Aqui está uma saída esperada:

dummy_output <- read.table(header=TRUE, text ="
                     ID Date        Measurement x_4_10_day Day_total x_4_10_night Night_total
                     12 2020.03.02  1           30         40        0            0
                     12 2020.03.03  2           0          0         45           75
                     13 2020.05.09  1           90         90        0            0
")

Todas as sugestões são muito apreciadas, obrigado!

E aqui estão os dados:

structure(list(date_time = c("2020.03.02 22:00:17", "2020.03.02 22:05:17", 
"2020.03.02 22:10:17", "2020.03.02 22:35:17", "2020.03.02 22:40:17", 
"2020.03.02 22:45:17", "2020.03.02 22:50:17", "2020.03.02 22:55:17", 
"2020.03.02 23:00:17", "2020.03.02 23:05:17", "2020.03.02 23:10:17", 
"2020.03.02 23:15:17", "2020.03.02 23:20:17", "2020.03.02 23:25:17", 
"2020.03.02 23:30:17", "2020.03.02 23:35:17", "2020.03.02 23:40:17", 
"2020.03.02 23:45:17", "2020.03.02 23:50:17", "2020.03.02 23:55:17", 
"2020.03.03 00:00:17", "2020.03.03 00:55:17", "2020.03.03 01:00:17", 
"2020.03.03 01:05:17", "2020.03.03 01:10:17", "2020.03.03 01:15:17", 
"2020.03.03 01:20:17", "2020.03.03 01:25:17", "2020.05.09 08:39:32", 
"2020.05.09 08:39:32", "2020.05.09 08:39:32", "2020.05.09 08:39:32", 
"2020.05.09 08:39:32", "2020.05.09 08:39:32", "2020.05.09 08:39:32", 
"2020.05.09 08:39:32", "2020.05.09 08:39:32", "2020.05.09 08:39:32", 
"2020.05.09 08:39:32", "2020.05.09 08:39:32", "2020.05.09 08:39:32", 
"2020.05.09 08:39:32", "2020.05.09 08:39:32", "2020.05.09 08:39:32", 
"2020.05.09 08:39:32", "2020.05.09 08:39:32"), id = c(12L, 12L, 
12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 
12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 
13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 
13L, 13L, 13L, 13L, 13L), x = c("7.55", "4.55", "4.55", "12", 
"12", "10", "10", "4.3", "", "", "4.3", "4.3", "4.3", "", "4.3", 
"12", "12", "12", "2", "12", "12", "", "8", "3", "3", "2", "2", 
"", "12", "10", "10", "4.3", "4.3", "4.3", "4.3", "4.3", "4.3", 
"4.3", "4.3", "12", "12", "12", "12", "12", "12", "12")), row.names = c(NA, 
46L), class = "data.frame")

Respostas

1 YBS Oct 11 2020 at 00:00

Eu adicionei id=14apenas valores noturnos ao seu dataframe. Talvez seja isso que você está procurando. Observe que os valores esperados não atendem totalmente aos seus requisitos.

df11 <- structure(list(date_time = c("2020.03.02 22:00:17", "2020.03.02 22:05:17", 
                             "2020.03.02 22:10:17", "2020.03.02 22:35:17", "2020.03.02 22:40:17", 
                             "2020.03.02 22:45:17", "2020.03.02 22:50:17", "2020.03.02 22:55:17", 
                             "2020.03.02 23:00:17", "2020.03.02 23:05:17", "2020.03.02 23:10:17", 
                             "2020.03.02 23:15:17", "2020.03.02 23:20:17", "2020.03.02 23:25:17", 
                             "2020.03.02 23:30:17", "2020.03.02 23:35:17", "2020.03.02 23:40:17", 
                             "2020.03.02 23:45:17", "2020.03.02 23:50:17", "2020.03.02 23:55:17", 
                             "2020.03.03 00:00:17", "2020.03.03 00:55:17", "2020.03.03 01:00:17", 
                             "2020.03.03 01:05:17", "2020.03.03 01:10:17", "2020.03.03 01:15:17", 
                             "2020.03.03 01:20:17", "2020.03.03 01:25:17", "2020.05.09 08:39:32", 
                             "2020.05.09 08:39:32", "2020.05.09 08:39:32", "2020.05.09 08:39:32", 
                             "2020.05.09 08:39:32", "2020.05.09 08:39:32", "2020.05.09 08:39:32", 
                             "2020.05.09 08:39:32", "2020.05.09 08:39:32", "2020.05.09 08:39:32", 
                             "2020.05.09 08:39:32", "2020.05.09 08:39:32", "2020.05.09 08:39:32", 
                             "2020.05.09 08:39:32", "2020.05.09 08:39:32", "2020.05.09 08:39:32", 
                             "2020.05.09 08:39:32", "2020.05.09 08:39:32", 
                             "2020.03.02 23:45:17", "2020.03.02 23:50:17", "2020.03.02 23:55:17", 
                             "2020.03.03 00:00:17", "2020.03.03 00:55:17", "2020.03.03 01:00:17" 
                             ), 
                      x = c("7.55", "4.55", "4.55", "12", 
                            "12", "10", "10", "4.3", "", "", "4.3", "4.3", "4.3", "", "4.3", 
                            "12", "12", "12", "2", "12", "12", "", "8", "3", "3", "2", "2", 
                            "", "12", "10", "10", "4.3", "4.3", "4.3", "4.3", "4.3", "4.3", 
                            "4.3", "4.3", "12", "12", "12", "12", "12", "12", "12",
                            "12", "10", "10", "4.3", "4.3", "4.3"),
               id = c(12L, 12L, 
                      12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 
                      12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 
                      13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 
                      13L, 13L, 13L, 13L, 13L, 14L, 14L, 14L, 14L, 14L, 14L)), 
               row.names = c(NA, 52L), class = "data.frame")

df11$xn <- as.numeric(df11$x)
df1 <- df11 %>% transform(xmin = ifelse((xn<4 | xn>10 | is.na(xn)),0,5 ),
                          xmint = ifelse(is.na(xn),-5,5 ))
df1$dateTime = as_datetime(df1$date_time, format = "%Y.%m.%d %H:%M:%S")
df1$mydate = as.Date(df1$date_time, format = "%Y.%m.%d %H:%M:%S")

df1$tm <- as.numeric(df1$dateTime)
df1$dts <- 86400*as.numeric(df1$mydate)

df2 <- df1 %>% group_by(id,mydate) %>% 
         transform(date = case_when(((dts-3600)<tm & tm<(dts+82800) )~paste0(mydate),((dts+82800)<=tm)~paste0(mydate+1) )) %>%
         transform(dayrnight = ifelse((tm>=(dts+25200) & tm<(dts+82800) ),'day','night' ) ) %>% 
         group_by(id,date,dayrnight) %>% 
         dplyr::summarise(x_4_10 = sum(xmin), total = sum(xmint)) %>% 
         pivot_wider(id_cols = c(id,date), names_from = dayrnight, values_from = c("x_4_10", "total")) %>% 
         mutate_if(is.numeric , replace_na, replace = 0) %>% 
         group_by(id) %>% mutate(measurement = row_number()) %>% 
         select(id,date,measurement,x_4_10_day,total_day,x_4_10_night,total_night)

> df2
# A tibble: 4 x 7
# Groups:   id [3]
     id date       measurement x_4_10_day total_day x_4_10_night total_night
  <int> <chr>            <int>      <dbl>     <dbl>        <dbl>       <dbl>
1    12 2020-03-02           1         30        40            0           0
2    12 2020-03-03           2          0         0           25          50
3    13 2020-05-09           1         50        90            0           0
4    14 2020-03-03           1          0         0           25          30

1 AnilGoyal Oct 11 2020 at 03:58

Levei algum tempo, mas provavelmente você quer isso

dados de amostra (mudaram um pouco porque a data / hora em 13eram todos iguais

df <- structure(list(date_time = c("2020.03.02 22:00:17", "2020.03.02 22:05:17", 
                             "2020.03.02 22:10:17", "2020.03.02 22:35:17", "2020.03.02 22:40:17", 
                             "2020.03.02 22:45:17", "2020.03.02 22:50:17", "2020.03.02 22:55:17", 
                             "2020.03.02 23:00:17", "2020.03.02 23:05:17", "2020.03.02 23:10:17", 
                             "2020.03.02 23:15:17", "2020.03.02 23:20:17", "2020.03.02 23:25:17", 
                             "2020.03.02 23:30:17", "2020.03.02 23:35:17", "2020.03.02 23:40:17", 
                             "2020.03.02 23:45:17", "2020.03.02 23:50:17", "2020.03.02 23:55:17", 
                             "2020.03.03 00:00:17", "2020.03.03 00:55:17", "2020.03.03 01:00:17", 
                             "2020.03.03 01:05:17", "2020.03.03 01:10:17", "2020.03.03 01:15:17", 
                             "2020.03.03 01:20:17", "2020.03.03 01:25:17", "2020.05.09 08:39:32", 
                             "2020.05.09 08:44:32", "2020.05.09 08:49:32", "2020.05.09 08:54:32", 
                             "2020.05.09 08:59:32", "2020.05.09 09:39:32", "2020.05.09 09:44:32", 
                             "2020.05.09 09:49:32", "2020.05.09 09:59:32", "2020.05.09 10:39:32", 
                             "2020.05.09 11:39:32", "2020.05.09 12:39:32", "2020.05.09 13:39:32", 
                             "2020.05.09 14:39:32", "2020.05.09 15:39:32", "2020.05.09 16:39:32", 
                             "2020.05.09 17:39:32", "2020.05.09 18:39:32"), id = c(12L, 12L, 
                                                                                   12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 
                                                                                   12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 12L, 
                                                                                   13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 13L, 
                                                                                   13L, 13L, 13L, 13L, 13L), x = c("7.55", "4.55", "4.55", "12", 
                                                                                                                   "12", "10", "10", "4.3", "", "", "4.3", "4.3", "4.3", "", "4.3", 
                                                                                                                   "12", "12", "12", "2", "12", "12", "", "8", "3", "3", "2", "2", 
                                                                                                                   "", "12", "10", "10", "4.3", "4.3", "4.3", "4.3", "4.3", "4.3", 
                                                                                                                   "4.3", "4.3", "12", "12", "12", "12", "12", "12", "12")), row.names = c(NA, 
                                                                                                                                                                                           46L), class = "data.frame")

Resultado Editado

library(tidyverse)
library(lubridate)

df %>% as_tibble() %>%
  transform(x = as.numeric(x), 
            date_time = as_datetime(date_time),
            id = as.character(id)) %>%
  mutate(d_n = ifelse(hour(date_time)>=7 & hour(date_time)<23, 'day', 'night'),
         Date = as.Date(date_time, format = "%Y.%m.%d %H:%M:%S"),
         valid_m = ifelse(x>=4 & x<= 10, 1, 0)) %>%
  mutate(valid_m = ifelse(is.na(valid_m), 0, valid_m)) %>% #valid measurements
  arrange(id, date_time) %>%
  group_by(id) %>%
  mutate(validm_d = as.numeric(lead(date_time)-date_time)) %>%
  filter(!is.na(validm_d)) %>%
  group_by(id, Date, d_n, valid_m) %>%
  summarise(x_tm = sum(validm_d)) %>%
  ungroup() %>%
  pivot_wider(names_from = d_n, values_from = x_tm, values_fill =0) %>%
  group_by(id, Date) %>%
  mutate(day_t = sum(day), night_t = sum(night)) %>% 
  filter(valid_m != 0) %>%
  group_by(id) %>%
  mutate(measurement = row_number()) %>%
  select(id, measurement, Date, x_4_10_day =day, x_4_10_total =day_t, 
         x_4_10_night =night, x_4_10_totaln = night_t)

resultado desejado

id    measurement Date       x_4_10_day x_4_10_total x_4_10_night x_4_10_totaln
  <chr>       <int> <date>          <dbl>        <dbl>        <dbl>         <dbl>
1 12              1 2020-03-02         50           60           20            60
2 12              2 2020-03-03          0            0            5            85
3 13              1 2020-05-09        235          600            0             0

Nesta solução, removi o último valor de cada medição, pois não tinha certeza de quanto tempo essa medição deveria ser feita. Você pode alterar o código de forma adequada. Basicamente, a última medição do 'dia' terminou às 2300 horas, portanto, os resultados da primeira linha deveriam ter sido 17 segundos a menos do que os mostrados.