1. Job 정보 가지고 오기.

 - 개발하는 중에 ItemReader, ItemProcessor, ItemWriter 등에서 job정보를 가지고 와야 할 때가 있다.

 - 간단한 소스 추가로 사용 가능하다.


2. 소스

private Long jobId;

@BeforeStep
public void getInterstepData(StepExecution stepExecution) {
JobExecution jobExecution = stepExecution.getJobExecution();
this.jobId = jobExecution.getJobId();
}

 - 비슷한 방식으로 확장해서 사용 가능 할 듯.


* 참고 링크


1. Multi Job

 - 한개의 Application 에서 여러개의 Job을 만들기


2. Job을 2개 만든다.

 1) 이전 글에서 만든 Job

/**
* Basic Configuration
*
* @author akageun
*/
@Configuration
public class BasicConfiguration {
private static final Logger LOG = LoggerFactory.getLogger(BasicConfiguration.class);

private static final String BASIC_JOB_NM = "BASIC_JOB";
private static final String BASIC_STEP_NM = "BASIC_TASKLET_STEP";

@Autowired
public JobBuilderFactory jobBuilderFactory;

@Autowired
public StepBuilderFactory stepBuilderFactory;

/**
* Basic Job Configuration
*
* @return
*/
@Bean(name = BASIC_JOB_NM)
public Job basicJob() {
//@formatter:off
return jobBuilderFactory
.get(BASIC_JOB_NM)
.incrementer(new RunIdIncrementer())
.start(basicTaskletStep())
.build();
//@formatter:on
}

/**
* Basic Step Configuration
*
* @return
*/
@Bean(name = BASIC_STEP_NM)
public Step basicTaskletStep() {
//@formatter:off
return stepBuilderFactory
.get(BASIC_STEP_NM)
.tasklet((stepContribution, chunkContext) -> {
LOG.info("Tasklet Run!!");

return RepeatStatus.FINISHED;
})
.build();
//@formatter:on
}
}


 2) 새로 만든 Multi Job


import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
* MULTI BASIC Configuration
*
* @author akageun
*/
@Configuration
public class MultiBasicTestConfiguration {
private static final Logger LOG = LoggerFactory.getLogger(MultiBasicTestConfiguration.class);

private static final String MULTI_BASIC_JOB_NM = "MULTI_BASIC_JOB";
private static final String MULTI_BASIC_STEP_NM = "MULTI_BASIC_TASKLET_STEP";

@Autowired
public JobBuilderFactory jobBuilderFactory;

@Autowired
public StepBuilderFactory stepBuilderFactory;

/**
* MULTI_BASIC Job Configuration
*
* @return
*/
@Bean(name = MULTI_BASIC_JOB_NM)
public Job multiBasicJob() {
//@formatter:off
return jobBuilderFactory
.get(MULTI_BASIC_JOB_NM)
.incrementer(new RunIdIncrementer())
.start(multiBasicTaskletStep())
.build();
//@formatter:on
}

/**
* MULTI_BASIC Step Configuration
*
* @return
*/
@Bean(name = MULTI_BASIC_STEP_NM)
public Step multiBasicTaskletStep() {
//@formatter:off
return stepBuilderFactory
.get(MULTI_BASIC_STEP_NM)
.tasklet((stepContribution, chunkContext) -> {
LOG.info("Tasklet Multi Job Run!!");

return RepeatStatus.FINISHED;
})
.build();
//@formatter:on
}
}


3. 실행

 1) application.yml 내 옵션 설정

spring:
batch:
job:
enabled: true

 - 위와 같이 설정할 경우 해당 App내에 모든 Job이 실행된다.


 2) application 내 옵션

spring:
batch:
job:
enabled: false
names: BASIC_JOB,MULTI_BASIC_JOB

- 위와같이 설정할 경우 names에 들어있는 Job들만 실행된다. (Job들은 콤마(,)로 구분된다)


 3) jar파일 실행

java -jar ./BATCH_STUDY.jar --spring.batch.job.names=BASIC_JOB,MULTI_BASIC_JOB


 4) intellij 실행


4. 결과


5. 전체소스보기 : 바로가기

1. 기본 세팅 해보기

 1) JOB 세팅하기

@Autowired
public JobBuilderFactory jobBuilderFactory;

/**
* Basic Job Configuration
*
* @return
*/
@Bean(name = BASIC_JOB_NM)
public Job basicJob() {
//@formatter:off
return jobBuilderFactory
.get(BASIC_JOB_NM)
.incrementer(new RunIdIncrementer())
.start(basicTaskletStep())
.build();
//@formatter:on
}

 2) STEP

(1) 기본 소스

@Autowired
public StepBuilderFactory stepBuilderFactory;

/**
* Basic Step Configuration
*
* @return
*/
@Bean(name = BASIC_STEP_NM)
public Step basicTaskletStep() {
//@formatter:off
return stepBuilderFactory
.get(BASIC_STEP_NM)
.tasklet((stepContribution, chunkContext) -> {
LOG.info("Tasklet Run!!");

return RepeatStatus.FINISHED;
})
.build();
//@formatter:on
}

(2) Tasklet

 - org.springframework.batch.core.step.tasklet.Tasklet 로 구현함.

 - 샘플소스

@Component
public class TaskletStep implements Tasklet {

@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {

LOG.info("Tasklet Run!!");

return RepeatStatus.FINISHED;
}
}

- 위 소스를 간단하게 Step 내 소스처럼 변경 가능하다.


2. 실행해보기

 - java -jar로 실행해보기

java -jar path/batch_file_name.jar --spring.batch.job.names=BASIC_JOB


 - intellij 에서 실행해보기


3. 소스 확인하기

 - 바로가기 

1. Spring Batch

 1) Spring batch란? 

  - Spring Batch는 Job과 Step으로 구성되어 있음.

  - 하나의 Spring Batch안에는 여러 Job이 존재 할 수 있고, 그 Job 안에는 여러 개의 Step 또는 Tasklet을 존재 할 수 있음.

  - Job -> Step -> ItemReader - ItemProcessor - ItemWriter


#https://docs.spring.io/spring-batch/trunk/reference/htmlsingle/#domain 에서 가져온 이미지 입니다.


  2) 장점

   - 간단하게 대용량 배치를 만들 수 있다.

   - 이미 만들어진 많은 모듈들을 사용해서 손쉽게 구현가능(CSV 파싱, DB에서 가지고 오기, S3 등에 파일업로드 등)

  

  3) 단점

    - 어렵다.(배우면 된다.)


2. Class 설명

  1) ItemReader

  - Step 안에서 데이터를 가져오는 역할을 하는 Class이다.

  - 하나의 아이템이 리턴되거나 null

package org.springframework.batch.item;

/**
* Strategy interface for providing the data. <br>
*
* Implementations are expected to be stateful and will be called multiple times
* for each batch, with each call to {@link #read()} returning a different value
* and finally returning <code>null</code> when all input data is exhausted.<br>
*
* Implementations need <b>not</b> be thread-safe and clients of a {@link ItemReader}
* need to be aware that this is the case.<br>
*
* A richer interface (e.g. with a look ahead or peek) is not feasible because
* we need to support transactions in an asynchronous batch.
*
* @author Rob Harrop
* @author Dave Syer
* @author Lucas Ward
* @since 1.0
*/
public interface ItemReader<T> {

/**
* Reads a piece of input data and advance to the next one. Implementations
* <strong>must</strong> return <code>null</code> at the end of the input
* data set. In a transactional setting, caller might get the same item
* twice from successive calls (or otherwise), if the first call was in a
* transaction that rolled back.
*
* @throws ParseException if there is a problem parsing the current record
* (but the next one may still be valid)
* @throws NonTransientResourceException if there is a fatal exception in
* the underlying resource. After throwing this exception implementations
* should endeavour to return null from subsequent calls to read.
* @throws UnexpectedInputException if there is an uncategorised problem
* with the input data. Assume potentially transient, so subsequent calls to
* read might succeed.
* @throws Exception if an there is a non-specific error.
* @return T the item to be processed
*/
T read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException;

}


 2) ItemProcessor

  - Step 안에서 가져온 데이터를 가공하는 역할을 하는 Class이다.

  - Chunk 사이즈 만큼 Item들이 List로 들어온다.

package org.springframework.batch.item;

/**
* Interface for item transformation. Given an item as input, this interface provides
* an extension point which allows for the application of business logic in an item
* oriented processing scenario. It should be noted that while it's possible to return
* a different type than the one provided, it's not strictly necessary. Furthermore,
* returning null indicates that the item should not be continued to be processed.
*
* @author Robert Kasanicky
* @author Dave Syer
*/
public interface ItemProcessor<I, O> {

/**
* Process the provided item, returning a potentially modified or new item for continued
* processing. If the returned result is null, it is assumed that processing of the item
* should not continue.
*
* @param item to be processed
* @return potentially modified or new item for continued processing, null if processing of the
* provided item should not continue.
* @throws Exception
*/
O process(I item) throws Exception;
}


 3) ItemWriter

  - Step 안에서 데이터를 쓰는 역할을 하는 Class이다.

package org.springframework.batch.item;

import java.util.List;

/**
* <p>
* Basic interface for generic output operations. Class implementing this
* interface will be responsible for serializing objects as necessary.
* Generally, it is responsibility of implementing class to decide which
* technology to use for mapping and how it should be configured.
* </p>
*
* <p>
* The write method is responsible for making sure that any internal buffers are
* flushed. If a transaction is active it will also usually be necessary to
* discard the output on a subsequent rollback. The resource to which the writer
* is sending data should normally be able to handle this itself.
* </p>
*
* @author Dave Syer
* @author Lucas Ward
*/
public interface ItemWriter<T> {

/**
* Process the supplied data element. Will not be called with any null items
* in normal operation.
*
* @param items items to be written
* @throws Exception if there are errors. The framework will catch the
* exception and convert or rethrow it as appropriate.
*/
void write(List<? extends T> items) throws Exception;

}


3. 간단 용어 설명

  1) Item 

   - 데이터의 가장 작은 구성 요소를 말함.


 2) Chunk

   - commit-interval

   - ItemReader 읽은 데이터를 Processor 통해 가공 한 후 ItemWriter 넘겨지는 갯수를 의미함.

   - 트랜젝션이 걸려 있다면 한 트랜젝션 안에서 처리할 Item의 수이다.


4. 옵션

     (1) 옵션 설명

- spring.batch.initializer.enabled : Spring Batch 실행시에 Database 내 Table create 등 실행여부

- spring.batch.job.enabled : Spring Batch 실행시에 Context내 모든 Job들 실행 여부

- spring.batch.job.names : 실행할 Job 리스트, 콤마(,)로 구분한다.

- spring.batch.schema : db 스키마 초기화 sql파일 위치 (바로가기)

- spring.batch.table-prefix : 테이블명 앞에 붙일 명칭


     (2) 설정

  - application.properties

spring.batch.initializer.enabled=false
spring.batch.job.enabled=false
spring.batch.job.names=test1,test2,test3
spring.batch.schema=org/springframework/batch/core/schema-h2.sql
spring.batch.table-prefix=tmp_


  - application.yml

spring:
batch:
schema: org/springframework/batch/core/schema-h2.sql
table-prefix: tmp_
initializer:
enabled: false
job:
enabled: false
names: test1,test2,test3


* 참고페이지

 - https://projects.spring.io/spring-batch/

 - https://docs.spring.io/spring-batch/3.0.x/reference/html/index.html

 - https://docs.spring.io/spring-batch/trunk/reference/html/listOfReadersAndWriters.html


+ Recent posts