Ventur's questions

Good morning, I am learning Spark and Kafka, I have a streaming application using the spark API for java . My problem lies in how to know what was the last offset read? For example, my application constantly reads from kafka, suddenly it crashes and messages arrive in the kafka topic. I want to make it so that when my Spark application wakes up, it continues reading from the offset that was read before the crash. With kafkaUtils if I put:

auto.offset.reset=earliest

It reads the entire topic for me, if I change it to latest it reads the records that arrive once the application starts, so it doesn't do what I want, since I would lose data from Kafka. Add a checkPoint , I understand that this works when a DStream is not finished reading, if it crashes you can continue with a job that was interrupted.

Configuration code:

Spark config:

return new SparkConf().setMaster(sparkDriverUtils.getSparkMaster())
                .setAppName(sparkDriverUtils.getSparkAppName() == null ? "name-not-set"
                        : sparkDriverUtils.getSparkAppName())
                .set("spark.default.parallelism", "2")
                
                .set("spark.yarn.maxAppAttempts", "1")
                .set("spark.yarn.am.attemptFailuresValidityInterval", "2h")
                            
                .set(DriverConstants.SPARK_MAX_CORES, sparkDriverUtils.getSparkMaxCores())
                .set(DriverConstants.SPARK_EXECUTOR_MEMORY, sparkDriverUtils.getSparkExecutorMemory()); 

Snippet for Kafka: Kafka Parameters:

kafkaParams.put(DriverConstants.KAFKA_BOOTSTRAP_SERVER, getBootstrapServers());// 1 or more brokers
        kafkaParams.put(DriverConstants.KEY_DESERIALIZER, StringDeserializer.class);
        kafkaParams.put(DriverConstants.VALUE_DESERIALIZER, StringDeserializer.class);
        kafkaParams.put(DriverConstants.KAFKA_GROUP_ID, getGroupId());
        kafkaParams.put(DriverConstants.KAFKA_AUTO_OFFSET_RESET, getAutoOffsetReset()); //earliest  o latest
    kafkaParams.put(DriverConstants.KAFKA_ENABLE_AUTO_COMMIT, getEnableAutoCommit()); //false



    JavaStreamingContext jssc = sparkConfigurationBuilder
            .buildJSC(sparkConfigurationBuilder.buildSparkConfiguration());
            
    jssc.checkpoint("C:\\tmp\\poc\\checkPoint");
    Map<String, Object> kafkaParams = sparkDriverUtils.getKafkaProperties();        
    Collection<String> topics = Arrays.asList(sparkDriverUtils.getTopics().trim().split(","));// 1 o more topics        


JavaInputDStream<ConsumerRecord<String, String>> stream = KafkaUtils.createDirectStream(jssc,
            LocationStrategies.PreferConsistent(),
            ConsumerStrategies.<String, String>Subscribe(topics, kafkaParams));

Greetings.

I am trying to insert a vectorized image in SVG format, but I can only see it in mozilla firefox, for IE and chrome it does not appear. I have looked for some documentation on svg in browsers, in many I see examples where shapes are drawn by placing coordinates, colors and the browser renders in svg image, but at home I have that svg image in a file.

Code :

   <div class= "spacioLogos">
         <p:graphicImage value="./images/logo_mexico.svg" width="120" height="55" styleClass="logoizquierda"/>

         <p:graphicImage value="./images/logo_mexico.svg" width="120" height="55" styleClass="img"/>                    

    </div>

The same is seen if I place the html img:

<img src="./images/logo_mexico.svg" width="120" height="55" class="responsive" />

Sample of how it is rendered in browsers: mozila firefox:

Google Chrome:

internet explorer edge:

Hello, I am doing an upload and download of a txt. When I click on "Download" the txt, it is downloaded, and I set some variables, later another panel must be rendered, my problem is that I download the file and the panel is not rendered, if I click on download again, then the panel is rendered. I hope you understand me .

Panel with buttons:

<p:panel styleClass="panelGridConsulta">
    <div class="panelGrid botonesFlujoCaptcha">
        <p:commandButton styleClass="estilo" value="Terminar"
                         actionListener="#{bean.reset}" ajax="false"/>

        <p:commandButton id="comdDwn" value="Descargar archivo"
                         actionListener="#{fileBean.resetValida}" ajax="false"
                         disabled="#{fileBean.status ==null or bean.statusBtnDescargarArchivo ==null}"
                         update="panelDwn" styleClass="estilo">
            <p:fileDownload value="#{fileBean.file}" />
        </p:commandButton>
    </div>
</p:panel>

Method in MB:

public void resetValida() {
    setStatus(null);
    setStatusDownload("Favor de seleccionar y validar un archivo.");
    terminarMasiva = "activo descarga";
    RequestContext.getCurrentInstance().update("terminaDescargaArch");

}

Panel that should be rendered once the txt is downloaded:

<p:panel id="terminaDescargaArch" styleClass="panelGridConsulta"
rendered="#{uploadFileBean.terminarMasiva != null}">
    <h:outputLabel styleClass="txtLabel izq3"
                   value="Test render after download:"/>
</p:panel>

It happens that I am using maven, I have configured 1 repository in my pom file, so that in case some dependency is not found in my M2, it looks for it in that repository, but it happens to me that, even if I have added the jars in my M2, maven it does not look for them and directly goes to look for a repository outside the local one.

Config in pom for repository:

<repositories>
    <repository>
      <id>xxxxx</id>
      <name>archiva</name>      
      <url>http://xx.xx.xx.xx/repository/internal/</url>
    </repository>
  </repositories> 

Do you know of a way to prevent maven from looking in another repository than the local m2?

It happens that I'm trying to indicate the encoding in a BufferedWriter, but I can't do it, my implementation : Sonar findBug : Reliance on default encoding

        BufferedWriter out = new BufferedWriter(new FileWriter(comunConstants.getRutaArchivo(),true));

I read that with Charset, but I get syntax error.

        BufferedWriter out = new BufferedWriter(new FileWriter(comunConstants.getRutaArchivoRegistro(),true),Charset.defaultCharset());

what am I doing wrong?

For reasons of requirement, I have to combine components of primefaces and an input file in pure html. Now my question is, can I have the html input inside a JSF form? And that practically the entire view is with PrimeFaces and from a bean process the file to be loaded with the input file component. Example of what I want:

<p:panel header="Subir TXT" id="panelDwn" style="border: none;" >
    <h:panelGrid columns="1" cellpadding="5" style="border: none;">
        <h:outputLabel style="color: #545454; font-size: 16px;" value="Seleccione el archivo a validar TXT:" />
    </h:panelGrid>
    <h:panelGrid id="panelDes" columns="2" cellpadding="5" style="border: none;">
        <!-- INPUT FILE  HTML -->
        <input type="file" name="archivoupload"/>
        <h:outputLabel id="lblStatus" style="color: #545454; font-size: 16px;" value="#{uploadFileBean.status}" />                          
        <!--h:outputLabel style="color: #545454; font-size: 16px;" value="#{uploadFileBean.statusDownload}" /-->
    </h:panelGrid>
    <br />
    <h:panelGrid columns="2" cellpadding="5" style="border: none;">
        <p:commandButton styleClass="basicbtngobmx" value="Regresar" actionListener="#{captchaComponent.reset}" ajax="false" />
        <p:commandButton id="comdDwn" value="Descargar archivo" actionListener="#{uploadFileBean.resetValida}" ajax="false" disabled="#{uploadFileBean.status ==null or captchaComponent.statusBtnDescargarArchivo ==null}" update="fileUp">
            <p:fileDownload value="#{uploadFileBean.file}" ajax="false" />                           
        </p:commandButton>
    </h:panelGrid>                          
</p:panel>

From behind, from a bean, call a servlet to upload and process the file and have a processed txt as output. I just have my doubt for this functionality.

In the code, I try to exemplify that the inputFile will be html and the rest components of primefaces.