Weekly Update
This report will be a little bit longer than normal cause it will comprehend the last week changes too. So let’s start.
-
Status: We have been accompanying the status and behavior of the
update_versions
alterations onCirlce
to check if any further modifications or bug removals would be necessary, with some minor configurations to thedeploy
phase of theversions
files, we could see the new approach working very well since it wasmerged
. As an extra task from my mentors, I added a profiler for theauto_tick.py
script to verify the usage of thebot
process and check for eventual optimizations we can do. -
Abstract: For the
new_update_versions
update, we decided to switch the actualupdate_version
function for the new one, as also update the relative test script which can be seen on this PR. With a big help of CJ and Matt we removed the configuration for the original CIupdate
process, and replaced theupdate_versions
subprocess with the new one. Which was actually removing theupdate upstream versions
fromrun bot
:- run: name: update upstream versions command: | cd cf-graph source activate run_env conda-forge-tick --run 2
and update the
populate versions
process with the following modifications, which was just removing the previousmain
function fornew_update_versions
ofcli.xsh
file and replace for themain
of theupdate_upstream_versions
: ```python- run:
name: populate versions
command: |
cd cf-graph
source activate run_env
conda-forge-tick –run 2
During the test week, we saw some missing updated files from the feedstock, so CJ commented with a possible solution, it was to reformat the way we were reading the feedstock `bad` information (follow the conversation [here](https://github.com/regro/cf-scripts/pull/1073)), this way we replaced this
python with node_attrs[“payload”] as attrs: if attrs.get(“bad”) or attrs.get(“archived”):
- run:
name: populate versions
command: |
cd cf-graph
source activate run_env
conda-forge-tick –run 2
with
```python
with node_attrs["payload"] as attrs:
if (attrs.get("bad") and "Upstream" not in attrs["bad"]) or attrs.get(
"archived",
):
Then we get it running smoothly for the next week, now we only need to look for some code optimizations (if needed)
and look out an process workflow explanation. Which comes forth with the idea of an
profiler for the bot process, I’ve opted for the cProfiles
as it
attends to our necessities and have a better usage in my opinion, so the basic alteration made can be seen with this
explanatory code:
def function(*args) -> any:
# start profiler
profile_profiler = cProfile.Profile()
profile_profiler.enable()
# Do something....
# stop profiler
profile_profiler.disable()
# human readable
s_stream = io.StringIO()
profile_stats = pstats.Stats(profile_profiler, stream=s_stream).sort_stats(
"tottime",
)
profile_stats.print_stats()
# get current time
now = datetime.now()
current_time = now.strftime("%d-%m-%Y") + "_" + now.strftime("%H_%M_%S")
# output to data
with open(f"profiler/{current_time}.txt", "w+") as f:
f.write(s_stream.getvalue())
which bumps the profiler information to a folder called profiler
and contains the information needed as the
respective run time (for inspection). These
Profiler LOG’s allows us to receive some feedback
for the time usage of the process and think of better ways to optimize then.
- Next: For the last meeting with my mentors and the conda-forge bot team, we decided on some topics for better time
and efficiency usage for the bot process,
profiling results, as
also discussed some future features for the bot structure. My tasks for next meeting will be:
- run the profiler on the pr json update steps too;
- make a PR to switch the bot to use rapidjson.
I also received the Google Summer of Code email for the last phase of the project and information regarding the final evaluation with the project resume and code work product. For this purpose I will also be working on the final work presentation during this last weeks.