How to create fiscal calendar in Incorta?(Part 3)

 The third part is the PySpark dataframe, this part is the most challenging part for me. Writing this part with PySpark is simpler and easier since we can get the data we want based on the previous result. For example, fiscal quarter seq and fiscal week of quarter can be built on the top of fiscal quarter number, and so on. I also practice how to use the window function in dataframe. For example, calculating fiscal year end date, fiscal month start date, fiscal month end date, and so on. 

  1. For getting the fiscal quarter number, we can use divide the week of year by 13. 
  2. For getting the fiscal month of quarter, we can depend on the fiscal week of quarter. When fiscal week of quarter <= 4, it is the first month of the quarter, and the month of quarter is 1. When fiscal week of quarter <=4+5, fiscal week of quarter is 2.  When fiscal week of quarter <=4+5+4, fiscal week of quarter is 3. Otherwise, when there are more than 52 weeks in a year, fiscal week of quarter is 3
  3. For getting fiscal year end date, fiscal month start date, fiscal month end date, we can use window function to partition data by fiscal year or fiscal month number to get the maximum or minimum value. 


Below is a screenshot of the dashboard in Incorta.



Comments

Post a Comment

Popular posts from this blog

How to create fiscal calendar in Incorta?(Part 2)

Image
This is the second part of the fiscal calendar MV, the Spark SQL part, through this part we can get day number, day name, fiscal week number, fiscal year start date, fiscal week end date, fiscal week of year, fiscal week start date, fiscal day seq, fiscal week date, fiscal day seq, fiscal week seq, fiscal year seq, fiscal day ago date, fiscal week ago date.  Divide the day of the year by 7, and if the remainder is 0, the week number is 7, which represents Sunday. Other week numbers are remainders.  Pushing the current date backward the days of the year is the fiscal year's start date.  To get the fiscal week of year. floor((doy-1)/7) . divided day of the year by 7 and remove the decimal. To get the fiscal week end date, first I get the fiscal week of the year floor((doy-1)/7) , and get the current number of days from the last day of the week through (((floor((doy-1)/7))+1)*7)-doy , then returns the week end date that is the num days after the date date_add(date, num_...
Read more

How to create histogram in Incorta use bin function.

Image
The  Ecommerce_Customer schema has four variables, Time On App, Time On Website, Length Of Membership, and Yearly Amount Spent. I want to see the distribution of these data. Incorta let me preview the data and show max and min data of each using its Preview function. Here are the steps I used to create a histogram in Incorta. First,  I used the bin function in Incorta divided into different levels. Here is the documentation for the bin function.        https://docs.incorta.com/4.5/r-bin Here is the result of the bin function. I divided the average of session length into 6 levels. If the length is less than 30, it will be labeled as 'SLV1', and if the length is greater than 30 but less than 32, it will be labeled as 'SLV2', by basically, according to the min and max value. I see the minimum value is close to 30, and the maximum value close to 38. I decided to use 2 minutes as the interval and created the formula using the bin function. I'm grouping...
Read more

Using Incorta and PySpark Linear Regression ML package to predict eCommerce Customer

Image
Project Overview I got a dataset from  kaggle.com .  Assumption: eCommerce company based in New York City that sells clothing online but they also have in-store style and clothing advice sessi ons. Customers come into the store, have sessions or meetings with a personal stylist, then they can go home and order either on a mobile app or website for the clothes they want. We need to predict 'Yearly Amount Spent' Here are the features or attributes collected in the dataset: 'Avg__Session_Length' 'Time_on_App' 'Time_on_Website' 'Length_of_Membership' Step 1: Upload csv file in incorta. Upload the CSV file to Incorta, and add a file table in the schema named Ecommerce_Customer. Step 2: Read the Ecommerce Customer file Use PySpark to read the table named SparkTesting.Ecommerce_Customer. The CSV file loaded into Incota can be read into PySpark using  df=read("SparkTesting.Ecommerce_Customer") Step 3: VectorAssemblerTest Use...
Read more